Existing models for Internet Autonomous System (AS) topology generation make structural assumptions about the AS graph. Those assumptions typically stem from beliefs about the true properties of the Internet, e.g. hierarchy and powerlaws, which arise from incorrect interpretations of incomplete observations of the AS topology. In this paper we compare AS topology generation models with several observed AS topologies without making assumptions as to the relative importance of different topological characteristics. We find that although existing AS topology models capture degree-based properties well, they fail to capture the complexity of the local interconnection structure between ASes. We use a wide range of metrics including the weighted spectral distribution and make it available as toolbox1. We show that the shortcomings of existing models stem from underestimating the complexity of connectivity in the core due to incomplete understanding of collected data limitations, and narrow focus on particular aspects of the AS topology structure.