Toichi died (yes he DIED GOSHSO) when Kaito was 8 correct? So while Kaito had learned a lot of his magic from his father at that point, the rest he must have learned by himself. Imagine a young Kaito, determined to be the greatest magician in the world meticulously studying the videos taken of his father, with each watch he takes the tricks apart piece by piece and he realises something as he does... he feels no joy learning his father's tricks, in fact he almost feels like he's killing the magic that he loved. It wasn't the same as learning the secrets from his father's mouth or impressing him by showing how he figured out the latest magic trick he showed and challenged him to learn, no learning it this way hurt, it hurt too much, and yet he needed to know, needed to master them just as his father had before so he could one day become as great as him.
When he encounters people after his magic show wanting to uncover how its done, or even unravel the mystery for everyone who had previously wanted to be fooled, it takes him back to those days where he would slowly destroy his father's magic so he can recreate it. Critics, he calls them in his head. The murderers of magic.
So when Kaito encounters a little boy, a boy who should be too young to be this jaded and take pride in pulling his tricks apart in that smart and arrogant way, he labels him the same as he did the others, another critic.
And then the boy comes back. He deciphers another trick with a smug smirk directed at him yet there is a joy in his eyes, one Kaito doesn't see often in these types of people. Kaito realises, the boy isn't proud of figuring out his tricks because he enjoys killing the magic, no he loved the challenge and the mystery behind Kaito's magic. He enjoyed it just as much as he enjoyed figuring out how he did it.
Kaito is reminded of those times when his father challenged him to figure out his magic trick, how proud he felt when he finally managed to do so and how eager he was to tell his father that he had. This wasn't a critic who wanted to impress himself, no, he wanted to impress Kaito.