The Network Noisy Leaky Integrate and Fire equation is among the simplest model allowing for a self-consistent description of neural networks and gives a rule to determine the probability to find a neuron at the potential $v$. However, its mathematical structure is still poorly understood and, concerning its solutions, very few results are available. In the midst of them, a recent result shows blow-up in finite time for fully excitatory networks. The intuitive explanation is that each firing neuron induces a discharge of the others; thus increases the activity and consequently the discharge rate of the full network. In order to better understand the details of the phenomena and show that the equation is more complex and fruitful than expected, we analyze further the model. We extend the finite time bow-up result to the case when neurons, after firing, enter a refractory state for a given period of time. We also show that spontaneous activity may occur when, additionally, randomness is included on the firing potential $V_F$ in regimes where blow-up occurs for a fixed value of $V_F$.