CNS 2021 Tutorial:

Title: Recurrent Neural Networks dynamics and software implementation with Keras and TensorFlow.

Organizers:

Cecilia Jarne

Department of Science and Technology from the University of Quilmes and CONICET

Bernal, Buenos Aires, Argentina.

Correspondence: cecilia.jarne@unq.edu.ar or katejarne@gmail.com

Tutorial format: a half-day tutorial.

Description of the tutorial

This tutorial will help participants to implement and explore simple neural models using Keras [1], this year particularly with the focus on the implementation of Recurrent Neural Network (RNN) to perform temporal tasks. Such models are of great interest to different scientific communities, for example, Computational Neuroscience research and dynamical systems. Open-source frameworks dedicated to Machine Learning such as Keras [1] and Tensorflow [2] have produced significant changes in the development of technologies that we currently use. One relevant problem that can be approached using them is how to build the models for the study of dynamical systems and how to extract the relevant information to be able to answer scientific questions of interest related to brain modeling.

In this mini-course, we will study implementations of recurrent neural networks with Keras split into two sections: On one side, we will introduce the main keys of Recurrent Neural Networks, and on the other, the features of Keras showcasing some examples.

The Tutorial will include an introduction to modeling, discussion, and hands-on exercises. It will focus on using Keras to develop Neural Networks for rapid prototyping and simulate with TensorFlow [2] as backend. The tutorial will show how models can be built and explored using python scientific libraries. The hands-on exercises will demonstrate how Keras can be used to explore the dynamics of the network.

For this tutorial, you will need basic knowledge of NumPy, SciPy, and matplotlib.

To be able to carry out the tutorial, students need a laptop with Linux and these libraries installed or use google colab instead (https://colab.research.google.com/).

– Python

– Numpy

– SciPy

– Matplotlib

– Scikit learn

– TensorFlow

– Keras

I recommend the following sites where is explained the installation of the following packages that include a set of the named libraries and some additional tools:

https://www.anaconda.com/distribution/

https://www.tensorflow.org/install/

https://keras.io/

Tutorial schedule:

The first part consists of a Lecture on Recurrent Neural Networks and implementations with Tensorflow and Keras (2 hrs) then we will work on jupyter notebooks with google colab to implement simple and guided exercises 1 hrs approximately, depending on available time.

Slides for the Tutorial will be available at the date of the tutorial

References:

[1] Francois Chollet et al. Keras. https://keras.io, 2015.

[2] Martín Abadi, et al. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015.

[3] Deep Learning And Reinforcement Summer School (DLRLSS): https://dlrlsummerschool.ca/past-years/

[4] Deep Learning with Keras. Antonio Gulli Sujit Pal. Packt Publishing Ltd. 2017.

[5] https://www.anaconda.com/distribution/

[6] https://www.tensorflow.org/install/

[7] C. Jarne. What you need to know to train recurrent neural networks to make Flip Flops memories and more https://arxiv.org/abs/2010.07858