Solved – neural network decision boundary

neural networks

For the XOR problem, 2 decision boundaries are needed to solve it using 2 inputs neurons, 2 hidden neurons, 1 output neuron. From the book "Neural Network Design" on page 84, the author says that "each neuron in the network divides the input
space into two regions."

In the xor network, there are 3 neurons excluding the inputs. Therefore 3 decision boundaries should be drawn. But most books show only 2 decision boundaries for the xor problem. Can someone explain why is it so?

enter image description here

Best Answer

But there are 3 decision regions:

enter image description here

For example:

  • The first neuron splits the upper left blue input from the rest
  • The second neuron splits the lower right blue input from the rest
  • The output neuron splits the result into red area or blue area

Each neuron splits the input into one of 2 classes. Refer to Chapter 11 of that book for more detail.

For those interested, below is python code used to generate the plot. Not using neural-networks, just plotting a fabricated decision boundary:

import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
import numpy as np

boundary1 = np.linspace(-.2,1.2,100)
plt.plot(boundary1,boundary1+0.4,c='black')
plt.plot(boundary1,boundary1-0.6,c='black')
plt.scatter(1,1,c='red',s=100)
plt.scatter(0,0,c='red',s=100)
plt.scatter(0,1,s=100)
plt.scatter(1,0,s=100)
plt.fill_between(x=boundary1,y1=boundary1+.4,y2=boundary1+3,alpha=.2,color='blue')
plt.fill_between(x=boundary1,y1=boundary1-.6,y2=boundary1-2,alpha=.2,color='blue')
plt.fill_between(x=boundary1,y1=boundary1+.4,y2=boundary1-0.6,alpha=.2,color='red')
plt.xlim(-.2,1.2)
plt.ylim(-.1,1.1)
plt.show()
Related Question