Scientific modeling has been advocated as one of the core practices in recent science education policy initiatives. In modeling-based instruction (MBI), students use, construct, and revise models to gain scientific knowledge and inquiry skills. Oftentimes, the benefits of MBI have been documented using assessments targeting students' conceptual understanding or affective domains. Fewer studies have used assessments directly built on the ideas of modeling. The purpose of this study is to synthesize and examine modeling-oriented assessments (MOA) in the last three decades and propose new directions for research in this area. The study uses a collection of 30 empirical research articles that report MOA from an initial library of 153 articles focusing on MBI in K-12 science education from 1980 to 2013. The findings include the variety of themes within each of the three MOA dimensions (modeling products, modeling practices, and meta-modeling knowledge) and the areas of MOA still in need of much work. Based on the review, three guiding principles are proposed for future work in MOA: (a) framing MOA in an ecology of assessment, (b) providing authentic modeling contexts for assessment, and (c) spelling out the connections between MOA items and the essential aspects of modeling to be assessed.