the books kinda did say what happened to them... not too explicitly though
Gilly was crying. “Me and the babe. Please. I’ll be your wife, like I was Craster’s. Please, ser crow. He’s a boy, just like Nella said he’d be. If you don’t take him, they will.”
“They?” said Sam, and the raven cocked its black head and echoed, “They. They. They.”
“The boy’s brothers,” said the old woman on the left. “Craster’s sons. The white cold’s rising out there, crow. I can feel it in my bones. These poor old bones don’t lie. They’ll be here soon, the sons.”
