We report on a long-term, large-scale study of a one-semester, conceptually based, introductory astronomy course with data from more than 400 students over three semesters at the University of New Mexico. Using traditional and alternative assessment tools developed for the project, we examined the pre- and postcourse results for Fall 1994, Spring 1995, and Fall 1995. We find our results are robust: novice students show large, positive gains on assessments of conceptual understanding and connected understanding of the knowledge structure of astronomy. We find no relationship between course achievement and completion of prior courses in science or math; we do find a small to moderate relationship between students’ science self-image and course achievement. Also, we detect little change over each semester in students’ mildly positive incoming attitudes about astronomy and science.