We’ve been talking a lot about AI and its future and yet we didn’t discuss what it is exactly. Here, I would like to briefly present one of the most impressive tools of Artificial Intelligence - neural networks.
Shortly, neural network is mathematical function consisting of matrices which takes an input, usually it is set of features e.g. pixels of an image, and than by means of various mathematical operations on the input and those matrices the network generates a result. One matrix inside of a net can be referred as a layer. Despite that neural nets were invented back in 1960s they were not that popular due to their massive computational consumption.
Shortly, neural network is mathematical function consisting of matrices which takes an input, usually it is set of features e.g. pixels of an image, and than by means of various mathematical operations on the input and those matrices the network generates a result. One matrix inside of a net can be referred as a layer. Despite that neural nets were invented back in 1960s they were not that popular due to their massive computational consumption.
Pic 1. Brain activity.
After almost half of a century they become extremely famous and all thanks to GPU(graphics processing unit). As it is well known, GPU is used for different graphical tasks like image/video rendering due to its parallel processing architecture. In other words GPU is able to perform multiple calculations (mostly on matrices) at the same time. To note, using GPU can speed up training process in 30-50 times !
Nowadays neural networks are used in numerous of tasks. They show state of the art performance in image classification, speech recognition, and the most satisfying in my opinion language translation.
For simplicity I’ll present an example of image classification. For example you want to classify handwritten digits from 0 to 9. Firstly, you need to gather dataset. One of the difficulties working with neural nets is that they need to “see” a lot of examples in order to “memorize” them. By a lot I mead hundred of thousands. Fortunately, for this example, there is such dataset called MNIST. It has 60 000 (6 000 per one digit) examples for training process and 10 000 (1 000 per one digit) for evaluating the network performance.
Pic 2. Image representation in form of matrix of 28 by 28 pixels with normalized values which means that values in the matrix should be between 0 and 1, wheres RGB format has values between 0 and 255.
Simplified training process looks like this:
- First, we present an image to the network (in Pic 3. it’s a vector with length 784) and the label corresponding to the image. The label is just another vector with length of 10 (as the number of classes) and all values as zero except one.
For example label for digit 0 is [1,0,0,0,0,0,0,0,0,0], for 5 [0,0,0,0,0,1,0,0,0,0].
9 - [0,0,0,0,0,0,0,0,0,1]
- Then goes feed forward pass. We multiply image vector, let say with digit 5, by the first hidden layer (as you may remember layer is just a matrix). As the result we get a vector which then multiplied by another layer. In the end we obtain the last vector which length corresponds to the number of classes and some values e.g :
[0.01, 0.31, 0.091, 0.27, 0.02, 0.12, 0.082, 0.07, 0.12, 0.16 ], although we are expecting to get the vector [0,0,0,0,0,1,0,0,0,0]. Our task is to tune values of hidden layers to get output vector with values similar to the label. In reality, you will never get an ideal result. Eventually, for digit 5 it will be something like:
[0.01, 0.001, 0.0091, 0.07, 0.02, 0.92, 0.082, 0.07, 0.002, 0.006 ] - Then we have back propagation. Backprop is the process which tunes values of this hidden layers (matrices) based on an error which occurred after feed forward pass.
In pic. 3 there is an example of feed forward pass.
Pic 3.
Passing all images through the network is called an epoch. After 5 -10 epochs, in this example, we can reach accuracy around 90%. The record for the network is around 98.5% when a person can recognize 96%.
Right now I’m working with the neural machine translation and I’m totally amazed by its performance. In a classification task it is more or less coherent of how it works. But in the translation task it’s much more difficult. For good translation, neural network needs to understand the context of a sentence. This year, Google Brain presented a state of the art architecture Transformer. The main feacher of the network that it can actually pay attention to the specific words and understand the context of a given sentence.
For instance in sentences:
The animal didn’t cross the street because it was too tired.
And
The animal didn’t cross the street because it was too wide.
Transformer model can understand what “it” we are considering, either
“it” is animal or street. In my opinion it is mind-blowing .
“it” is animal or street. In my opinion it is mind-blowing .
In conclusion, as we can see neural nets is a really powerful tool. What I’ve tried to explain above is a toy example, current architectures have hundreds of layers and dozens of its types. They really help to automate hundreds of daily tasks such as face recognition in iPhones, car number plate recognition, translation, voice recognition etc. I guess in the future they’ll become even more advanced.
1. What do you think about neural nets.
2. Did you change your mind about AI after this post.
3. How could you use a neural network in order to solve your daily problem ?
The best explanation of neural nets on the web:
https://ai.googleblog.com/2017/08/transformer-novel-neural-network.htmlhttps://www.youtube.com/watch?v=aircAruvnKk
Comments
I agree with the statements presented in the post, so I didn’t change my mind about AI after reading it.
In the terms of daily problems, I hope that further development in neural networks and robotics will allow to create machines which will be able to perform some daily, annoying human tasks. In my opinion robots similar to these from I, Robot movie would be great. They would allow humans to spend more time on much more interesting activities.
I absolutely agree with your opinion. Each month famous universities, companies publish new articles about some small break throughs in neural network architectures or optimization tasks. Even though image classification has been developing for almost a decade, neural machine translation is brand new field it is around 2-3 years and we already having out standing results.
But what we all tend to not think about are ramifications of privacy of data even developers dont think about for example legality of their training sets and data, while operational neural network usually is taken care of with such ramifications the development and training part is often overlooked.
If it was easier to build and train neural nets i could employ one to gather information for me , or analyze and adjust things in my life on the fly so much uses yet so far away to using neural net for them due to complexity.
Neural networks are a very interesting and useful field in today’s technology. I value them and their applications, although I’m not very experienced with them.
I especially like the applications of neural networks in computer vision (e.g. the ability to search photos by their content) and behavioural predictions (e.g. the ability for my phone to suggest actions for me).
I think you mistook neural network for artificial intelligence in your second question since your article hasn’t mentioned anything about artificial intelligence at all. This post has changed my attitude towards neither ML nor AI.
No, I haven't changed my mind about artificial intelligence after this post.
In my opinion, we can use a neural network in smart homes with Internet of Things devices. An intelligent home can recognize the face and open the door. Knowing our day plan earlier, he can start heating an hour earlier and prepare a drink and a meal for us. Depending on the temperature outside and the weather, adjust the lighting and the temperature inside.
Does not leave feeling that the science about neuronets is in a condition of cosmology now about 300 years ago when people only began to make catalogs of what was seen and to think out names for the happening processes. There are several parameters because of which today's neuronets even remotely cannot be compared to AI. What not when there is a need at accurate installation of quantity of layers and each of them has specialization. The reason also consists in it that most to decide how many layers and what are necessary for it. Tasks depending on complexity can manage denial, in couple of synapses and so on in process of increase. At the heart of network there has to be a mechanism of its complication in case of need and simplifications in case of simplification of a task and in a basic complete set. Otherwise this highly specialized decision under specific objectives.
I think that neural network can help us to get most important news for us and also thing that do google with their AI for camera is amazing and in future we will be able to do professional photos without special knowledge about photography.
My thoughts on neural networks have not changed much upon reading this article, I guess I share the sources I read with the author of the blogpost hence we share similarities in our view of neural nets.
3. I think I'd use them to monitor the growth speed of a plant given different quantities of life enabling substances such as water fertilizer and light.
I love the neural networks and everything that connected with them. And also I’m a data scientist, so I’m working with different types of NN on everyday purposes.
I did not change my mind about AI after this post. But I changed it when I was hearing the same information in my academy lectures in PJATK(‘NAI’). After that lectures, I have decided to start learning AI deeper and as a result of this, I become a data scientist.
When I’m thinking about my daily problems that could be solved using NN nothing is coming into my head, because they all solved, but for some of the solutions, I don’t have money. And I’m working on one NN that will solve my ‘cooking’ problem and it will be my graduate work also.
My opinion about that will never be changed because as I said before AI would destroy whole humanity in the future if we didn't stop.
About how AI and neural networks could solve my daily problems. I don't think that I need something for that purpose. I can solve problems by myself and I always recommend this feature to other. It really helps to become an adult and independent person.
In my opinion, I could use AI to help me deal with matters that usually take a lot of time to embrace. I would like someone who would understand my command and do it without problems.
2. No, I did not.
3. I do not have any problems I can solve with the help of the neural network.
I think neural networks are very good tool for some tasks. Thanks to the increasing computing power, they are becoming more and more popular and more easy to use due to frameworks like TuriCreate which adjust most parameters needed for training automatically.
2. Did you change your mind about AI after this post.
No, but I'm currently writing my thesis connected with neural networks, so I know the basics.
3. How could you use a neural network in order to solve your daily problem ?
Object detection using neural networks can be used to help people with low vision or completally blind.
To be honest I also had an idea to train a network for music suggestion. I have a hobby as mixing music. And in order to find a decent record it can take few hours of listening to hundreds of other songs.
Although, I guess you haven't paid your attention enough, because in the very beginning of the article I said that a neural network is a tool of AI which means it's not AI itself.