Most of us can't remember anything from before the age of three, but a new study suggests that, rather than being permanently lost, these memories could just be lying dormant in our brains, ready to resurface under the right conditions.
The results are based on research with young rats, but scientists think something similar could be happening in humans. After all, we retain plenty of knowledge from our earliest days, like how to walk and talk, and so our childhood memories might still be locked away somewhere.
The childhood memories we adults have difficulty recalling are what's known as episodic memories - specific associations about the who, what, when, where, and why of events.
We form these episodic memories all our lives, but many of us can't recall them before the age of two to four - a phenomenon called childhood amnesia, which other animals also experience when they're very young.
Researchers from New York University worked with 17 day-old lab rats - the equivalent of two or three-year-old children - and trained them to associate one side of a box with the feeling of a mild electric shock.
While the association was quickly forgotten due to their young age - the rats returned to that side of the box 24 hours later - the team was able to get the rats to 'remember' the association with another shock later on.
"Here we show that in rats an experience learned during the infantile amnesia period is stored as a latent memory trace for a long time," the team explains. "Indeed, a later reminder reinstates a robust, context-specific, and long-lasting memory."
The researchers also looked closely at activity in the hippocampus of the young and old rats - an area of the brain strongly associated with the long-term formation of memories - both before and after learning.
They found that when the hippocampus wasn't as active, the young rats weren't as good at forming latent memories and recalling them later, suggesting that active learning is important if we're to hang on to our earliest thoughts.
"Early in life, while the brain cannot efficiently form long-term memories, it is 'learning' how to do so, making it possible to establish the abilities to memorise long-term," says one of the researchers, Cristina Alberini.
"However, the brain needs stimulation through learning so that it can get in the practice of memory formation - without these experiences, the ability of the neurological system to learn will be impaired."
The researchers identified a protein called brain-derived neurotrophic factor (BDNF), which helps to grow and maintain neurons, and say it appears to be particularly important in retaining memories for the long term.
While humans and rats both have this protein, it's not yet clear if would have have the same effect in our brains when it comes to resurfacing memories lost to childhood amnesia.
But if scientists are able to somehow replicate the same kind of infantile recollections in people - and that's a big if - it could open the door to years of memories that most of us could otherwise never relive.
"The study breaks new ground," write Andrii Rudenko and Li-Huei Tsai from MIT, who were not involved with the research, in a commentary in Nature Neuroscience.
"It shows that very early hippocampus-dependant memories in mammals are not lost but stored as latent traces that can be recalled later."
But the reaction has been mixed from other scientists as to how well the results of the rat experiment might ultimately translate to the minds of human beings.
"The work is very elegant, and tells us a lot about the development of the hippocampus, but it doesn't tell us anything about infantile amnesia," Patricia Bauer from Emory University told New Scientist.
While our own ability to recall forgotten childhood memories still needs to be investigated, the study opens up an intriguing possibility, as well as suggesting that an active, learning brain can help confirm memories when we're very young.
The study has been published in Nature Neuroscience.