Implementing Neural Networks With Micrograd In Python

Hi there! I'm Shrijith Venkatrama, founder of Hexmos. Right now, I’m building LiveAPI, a tool that makes generating API docs from your code ridiculously easy.

  
  
  Modeling a Neuron


In serious neural network implementations, we model the neuron in the following way:

Input x0 (axon)
Weight w0 (synapse)
1 "Influence" x0*w0 (dendrite)
Sum of "influences" = x0*w0 + x1*w1 + ... (cell body)
Bias b 

The above leads to the cell body expression:


  ∑(xi⋅wi)+b\sum (x_i \cdot w_i) + b ∑(xi​⋅wi​)+b

We also have:

Activation function - squashing fuction (tanh, sigmoid)


The output axon is then:...