We prove, in a purely combinatorial way, the spectral curve topological recursion for the problem of enumeration of bi-colored maps, which in a certain way generalize the notion of dessins d'enfant. Furthermore, we give a proof of the quantum spectral curve equation for this problem. As a corollary, we prove a recent conjecture due to Do and Manescu on enumeration of hypermaps and give a new proof for their quantum spectral curve result in this case. Then we consider the generalized case of 4-colored maps and prove the corresponding spectral curve topological recursion.