Like a lot of people, I was completely baffled when Needletail was revealed to basically be in purgatory and not the Dark Forest. But after rereading AVS, I have come up with a theory. I will also note that there are likely several factors involved here. And lastly, I am not condoning anything Needletail has done. Let's make that perfectly clear now. So let's go through the possible reasons.
1. Her age. While Needletail was around two yrs when she died, most of her bad decisions where made when she was a young apprentice. She was young and dumb, to be blunt. She didn't understand the true consequences of her actions because she simply wasn't mature enough to do so.
2. She was a product of her environment. Needletail's attitude wasn't unique. Most of the Clan were disloyal and not much better than rogues. Her attitude was likely taught by her parents, as both of them willingly joined the rogues. All she did was take it one step further by actively inviting the rogues onto ShadowClan territory.
3. She tried to redeem herself while still alive. This is a key one. She showed remorse while still alive. This showed that she wasn't entirely evil and did have a conscience. In StarClan's eyes, this made her worth redeeming.
4. She is a natural leader. Now this is why I think she wasn't sent straight to the DF. Needletail has shown multiple times that she is a natural leaders. She was the leader of the apprentices and led the revolt. She was the first to join the rogues and influenced others to do the same. She was constantly influencing Violetshine to do bad things. She is incredibly manipulative and very intelligent. Now imagine her in the DF. She would quickly gain loyal followers. While the DF lost key members during the battle, a bunch of new cats joined them after the battle. She could be the next Mapleshade, and possibly start a new army. (You know, maybe that's why Ashfur didn't get sent to the DF....Nope. I'll start a new topic for that.)
5. HC didn't want to get rid of their best character. Self explanatory.
I mean really, all of these probably factor in to some extent.
Edit: I forgot to mention the others. As for them, it was probably just the sheer number of them. The DF would have become pretty powerful overnight if all of the Darktail's followers got sent there.