Unscientific Beliefs

In the Astronomy Education Review there is an awesome article by Chris Impey, Sanlyn Buxner, and Jessie Antonellis about Non-scientific beliefs among undergraduate students. They polled over 11K typically freshman or sophomore students over 22 years(!) about their non-scientific (signifying beliefs not acquired by application of the scientific method). They conclude that 'nonscientific ways of thinking are resistant to formal instruction, changing surprisingly little over the course of a college career that typically includes three science courses. The level of basic science knowledge among undergraduates is only weakly coupled to attitudes towards pseudoscience, and it coexists with attitudes and beliefs that are faith-based.'

The nonscientific beliefs probed range from those that are (I'd argue) antiscientific (e.g., 'The positions of the planets have an influence on the events of everyday life.' or 'Some numbers are especially lucky for some people.') to attitudes towards science or discussions of science ethics (e.g., 'Nuclear power is an important energy source and its use should be expanded.' or 'We should devote more of our money and scientific resources to repair damage done to the environment.').

Some of their interesting results:

• The degree of nonscientific beliefs varied very little with the stage within undergraduate education or the number of science classes taken. Courses fulfilling distribution science requirements apparently do little to diminish nonscientific beliefs.
• There was very little difference between the mean science literacy scores of cohorts with high and low scores for unscientific beliefs (it accounted for only 4% of the science literature score variance). Having unscientific beliefs appears to have little effect on your science literacy(!)
• Quoting from the paper : 'Completion of the college science requirement leads to very little improvement in science knowledge score. Educators cannot be satisfied with a situation where one in three graduates thinks antibiotics kill viruses as well as bacteria, one in four thinks that lasers work by focusing sound waves, and one in five thinks atoms are smaller than electrons, are unaware that the Earth goes around the Sun, or that humans evolved from earlier species.'

While I don't see that this question is addressed directly in this paper, the fact that such a tiny fraction of science literacy variance correlates with unscientific beliefs implies that even having (what I'd argue) antiscientific beliefs does little to degrade the quality of the students' science literacy - this surprises me. And, the sobering realization that basic scientific literacy is essentially unaffected by college distribution requirement courses is pretty depressing. I'm still digesting this - this may influence how I teach next semester's class....

Google docs used for before-class activities

Having used Google docs for in-class activities and having found it really useful, I wanted to try to use it for out of class asynchronous preparation for a challenging class. The goal of the class was to start the process of synthesizing all of the physical processes they had been thinking about into a picture of how galaxy formation works.

I decided on the following plan:

• Have the students do an activity where they describe their understanding of how a dark matter halo of a given mass forms, how it gets its gas from the intergalactic medium, what kind of galaxy lives in the center of the halo, what lives in the subhalos, etc. [I know this is technical gibberish, apologies - those are good questions that help them assemble the pieces of information and physical processes that they need to know for understanding how the galaxies in that halo works]. I had three groups, each with a different halo (=galaxy) mass. Each group had a different page on the same google doc (to stop too much editing on a single page). 
• They were asked to fill their part of the document out before class asynchronously and collaboratively, and I kept an eye on this process, asking leading questions or offering clarifications. 
• I added a final section where they were invited to write their questions [I have an area for writing questions down in all pre-class assignments, but they've never been public to the whole group before; I think you want to have a certain level of trust in the class before doing this?]. I worked on answering their questions, which in some cases turned into a nice dialog.
• They spent 10 minutes at the beginning of class in their expert groups coming to a consensus and identifying common questions.
• Then we discussed as a large group their answers, drawing parallels and creating links between their three different examples. 
• One of the particular topics we discussed was used to segue into a mini-lecture on an important physical process that they had not explored yet that was relevant for just one of the cases, this was pre-planned but worked out quite naturally.

I felt like this use of Google Docs was also very worthwhile. Almost all students engaged well with the exercise, they were able to do their activity in increments, making progress in the areas they understood, asking questions and getting feedback [from peers and myself] in areas they were less sure about, and being able to take their time to come to a better and more complete understanding of how their galaxy works. They were very well prepared for class, and our group discussion was very efficient, focused on common concerns and high-level synthesis.

Google docs used in-class...

I was recently alerted to the use of Google docs as a very useful resource for collaborative or group note-taking or activities.

I wanted to describe our use of it as part of a jigsaw (or this link) activity. They prepared outside of class as pre-work to discuss the role of three physical processes (shocks from structure formation, stellar feedback, and AGN feedback) in setting the properties of the hot gas between galaxies. They then discussed that in their three expert groups for ~15 minutes. We then split into three mixed groups, and they taught each other about the importance of their physical mechanisms and then together answered some questions that brought together the different threads.

We used Google docs in this last mixed-group part; a representative from each group typed up their discussion on a page that I had pre-prepared with the questions and 'shared' with them (one page per group, so three identical pages in the google doc; this is good because then they can edit in peace and not be fighting constantly over editing). Then we brought the groups together and we talked through some of the stickier issues together.

What were the effects of using Google Docs here?

• The interaction between the students was much less spontaneous, there was a lot more silence or just one person talking as people typed stuff in. This was a weird dynamic, that we hadn't had before during group discussions, and I was a little perturbed by it. I don't know that this is enough to outweigh the disadvantages, but it is definitely a minus of this idea.
• There were permanent records of their thoughts; they after class told me that this aspect was very valuable.
• I monitored the document as it built up, and I could insert comments, etc, to help guide the groups in their discussion of an issue. This was kind of cool, it was a different way of being able to simultaneously interact with all the groups.
• The use of one page per group allows them to build up their group answers without too much outside influence. When I've used this in faculty development classes where there are multiple groups per page, there are weird edit wars and you spend little time talking and a lot of time looking at other people in other groups typing in answers.
• They were able to read quickly over the contributions from the other groups at the start of the wrap-up, and this rapidly got them onto the same page, helped clarify some issues that other groups had understood well, and elevated the wrap-up to issues that they all had challenges with.
• I was able to then edit the document after class, inserting some clarifications or other thoughts. I don't know if this was thought to be valuable, but I think it might be, as after group discussions they are often nervous about whether they know the 'right' answer or just what each other thought.

To summarize, it seemed to me that some of the biggest disadvantages of the jigsaw (no or fragmentary records of what was discussed, everyone's anxiety about different groups having reached different stages of understanding, lack of an instructor-blessed right answer) were partially alleviated by use of Google docs as a collaborative tool, at the (perhaps considerable) cost of the spontaneity of the activity.

[update : we have used this again in class for the same purpose. This time it was more natural, as everyone was used to it, there was less loss of spontaneity of the activity (but still some loss). I am noticing though that there is no real activity on this after class, i.e., there is little engagement with the document after class finishes, except for things that I add. I might try to figure out how to encourage this.]

Lecturing!

I did it; I had to lecture.

All throughout the preparations I was asking myself - am I really doing this for the right reasons? Is this just because I was too lazy/behind/unimaginative to carefully construct a pre-class podcast or resources sheet and then have a more active class?

Galaxy Merger NGC 2623, as seen by HST.

I did try to hold myself to the reasons to lecture discussed in an earlier post. I'm into a part of the course where the course text is weak. We are exploring the the physics of galaxy formation, dark matter growth, dynamical friction, violent relaxation, gas cooling, star formation and stellar feedback. These processes are important at many stages of galaxy formation, and all of them are relevant for galaxy merging, see left.

Other available resources are incomplete or overly technical/detailed (papers or scary textbooks), and I needed to craft a 'middle way' that built intuition but is reasonably complete. For a couple of sessions I had small lectures coupled with some activities (e.g., them trying to figure out how to explain phenomena to a beginning undergrad as an attempt to synthesize the concepts by being put into their own words). Last class was all lecture(!)

But, lecture is different from when I taught this class traditionally two years ago. The students are very used to interrupting, asking questions; challenging assumptions. I still try to sit near the middle of the side of the table, which I think keeps the atmosphere less formal, more like a journal club. They're still grappling with the material during class. I've still got some pre-class and in-class assessment to do to see how effective this sequence of activities has been, but I feel like this has been a reasonably successful foray into small amounts of lecturing without ruining the active overall atmosphere of the class.

Misconceptions

There are some lovely articles in The Astronomy Education Review about common misconceptions about cosmology and gravity. Since understanding is related to the building up of a conceptual framework, the structure that is already there is crucially important. If there are deep-seated misconceptions that go unaddressed, many of our efforts to help students construct the correct mental model are doomed to failure.

Freefall is an awesome source of misconceptions...

Gravity is a nasty one. We experience the effects of gravity, so from early childhood we construct mental models for gravity. Kathryn Williamson and Shannon Willoughby from Montana State University (one of the very best centers for astronomy education research in the US) carefully constructed a set of illuminating questions probing the concepts in gravity that natural scientists valued the highest (including the universality of gravity; that gravity depends on mass and distance alone and not things like composition, magnetism, etc; gravity can't be blocked by intervening bodies; etc.). They then obtained open-ended written feedback about these probing questions from several hundred students, analyzing the results and conducting some follow-up interviews.


Some of the most prominent misconceptions that emerged were (quotations from the article):

• Boundaries, e.g., 'the surface of a planet, the edge of the atmosphere, or even an orbit ... may represent a casing that encloses a planet’s gravity, or a “check point” where gravity either disappears or diminishes. '
• 'An unexpected misconception is that the objects that orbit a planet can act as indicators of its surface gravity. ... the “orbiting indicators” concept is generally applied in two opposing ways: (a) A planet’s gravity must be stronger to reach out and hold objects that are far away, or (b) A planet’s gravity must be stronger to pull objects close to it. '
• 'Questionnaire results overwhelmingly support the well-documented ... misconception that gravity is confounded with magnetism, rotation, and atmospheric pressure.'

Other misconceptions include that only heavy objects can interact gravitationally or that denser objects have more gravity (at the same distance from the center).

This kind of study is an incredibly valuable tool (in principle) for designing activities to counter these misconceptions, and I look forward to including it in my teaching next semester. 

Things I desparately need to avoid doing...

I had the chance to give a colloquium last week, and what was really neat was I had the chance too to talk with a number of (primarily grad student teaching assistants) about learner centered and active
teaching, both at a meeting but also (more importantly) at dinner. The main messages that I took away from the discussions were :

1. The importance of not forgetting about the diversity of learning styles,
2. How crucial it is to give students access to the depth and level of materials they need (and by need, I think I mean successfully complete the assessments of the course, AND are left with the skills and knowledge they need for the next course in the sequence or curriculum, if the latter is a relevant consideration for one's course). This is not to belittle the importance of giving them a sequence of opportunities to practice the skills they will need also (that just wasn't the point during this conversation).

The most important insights were those from the graduate student instructors who had both been through active learning courses, and now were seeing things from an instructor's perspective also.

Two stories stuck in my mind. In the first, the student expressed the opinion that they had benefitted greatly from an early-stage inquiry-based course (where much of the course was them undertaking and solving an authentic problem with the assistance of the instructor), and what they learned there they remembered well. The problem was that this course was a prerequisite for additional courses, and content that was required by the second course had been removed from the first to create the room to make the course inquiry-based. The student had to make this content up themselves from studying from books.

The second was more concerning still. A department had converted many of its courses to an active format. A number of things were described to have gone wrong in different cases. In some areas of the curriculum good books were inadequate/non-existent, and with an absence of good notes (which the student attributed to a lack of lectures, I don't see that this needs to be the case, good notes could have been provided in other ways) it was very hard to meaningfully learn. In other areas, assessment was lacking(!) or completely mis-matched (usually much too involved, and classes were going over simpler content). An argument that was made is that students by the time they are majoring in a field have constructed their own 'active learning' support group already for asking questions, doing homeworks, etc., and that in those circumstances using class time to communicate the professor's structured take on the field is the most important goal.

I am reminded of the section in McKeachie's Teaching Tips where the reasons to lecture are discussed:

• presenting the most up to date view
• summarizing/synthesizing materials from scattered sources
• adapting material to student needs
• helping students read more effectively by creating a conceptual framework
• focusing on key principles/concepts/ideas
• modeling how to solve a problem like an expert
• communicating enthusiasm

At least for me, all this is food for thought....

Learning...

I'm in a class from the Center for Research on Learning and Teaching, a session on flipping the classroom. For the second session, they did a lovely job of identifying a couple of very short, snappy discussions of valuable elements of a flipped classroom: just in time teaching and peer instruction. They asked us to prepare to teach the rest of the class about a topic we had been assigned (in preparation for a jigsaw, I suppose) and pose some questions before class - putting us in the position of students in a flipped classroom.

From the instructor's perspective, I've been learning a lot over the last sessions, and in preparation for the next couple of sessions. On one hand, I've been learning a lot about just in time teaching from a practitioner's perspective - the importance of posing questions that are central to the learning goals (it's easy to ask the wrong question...), how the answers to the questions can dominate the shape of the classroom session, and how if the question is good it can help illuminate misconceptions or areas that aren't clear and help pave the way for learning that feels relevant (I hope, this is from my perspective and not from the student's perspective).

We've been talking about how the fact that there is such a huge range in galaxy luminosities (over a factor of 10 million range in luminosity, or energy emitted by the galaxy) coupled with the huge range in distances to galaxies leads to the inevitable conclusion that in an image such as the Hubble Ultra Deep Field (part of this is shown on the left), almost all the galaxies you see are among the brightest galaxies; there are many many thousands or millions of galaxies that should be in the image that are so faint that they are not detected, even in the deepest optical image ever taken. The pre-class activity was designed to work through an example of this effect, and we discussed this in class in response to that pre-class activity; we also have a homework designed to help towards internalizing this concept (and give the students a chance to use real data to determine and explore some of the relationships they are learning about).

This is encouraging - what is less encouraging is that I *still* talk too much in class, and I am becoming concerned that my approach to the course logical flow (which I took for what I thought were good reasons) may be making it harder for the students to navigate the course, because I've jumped around the book a lot (I wanted a galaxy evolution first approach, not a Milky Way first approach; perhaps more on this in a later post). I made an attempt to document better the flow of the course, but I feel like there is more I can do but I've not quite put my finger on it yet.

Engagement

Something wonderful happened in class. I am not sure why; perhaps it's because they've been thinking about the properties and motions of stars in galaxies for almost three weeks now, perhaps it's because we've explored the issue in depth using both Sparke and Gallagher (the very good course text) and Blanton and Moustakas (a splendid review). A contributing factor may be that I sat in a different spot, not near the board (we all sit around a conference table, and usually they're at the other end and I'm near the board, I moved to be away from the board and amongst them). It is goofy that sitting in a different spot might help change the atmosphere, but I think it might have? I also talked a lot less than usual, by design. I'm trying to talk as little as possible, but boy, I'm a blabbermouth.

The responses to the pre-class questions were excellent, and I had the chance to change what I thought we'd do in class to adapt to those questions - we talked about how to read papers for a while. Then we spent a good deal of time on a pre-class question that was done a little less well (perhaps because it was trying to connect concepts from different class sessions together, I think also because I worded it poorly), and in that activities students took the initiative, took risks by attempting unfamiliar problems, built on each others' answers, managed to synthesize concepts from different class sessions, and checked on their answers themselves (self-correcting).

NGC 5866, a lenticular galaxy (HST/NASA/ESA).

Then, we talked about lenticular galaxies. If you want to start a fight in a room full of astronomers who study galaxies, just ask one of them to tell you how lenticular galaxies form, stand back and watch the fireworks. They share a number of properties with spiral galaxies and elliptical galaxies, and I won't get into the details for fear of starting a fight about how they form.

Many of them identified in their responses to the pre-questions that they weren't sure how they formed, so we talked about it. How glorious it was! Someone said, 'well, they're faded spiral galaxies.' And someone else countered, 'but they've got larger bulges than spirals, they can't be faded spirals!' They were citing evidence, putting facts together, presenting contradictions, connecting them to other galaxy types, challenging (justifiably) the evidence cited by each other (and myself), posing to first order all the important questions of lenticular galaxy research. Everyone spoke today, some more than others, but everyone spoke.

Could well be the best teaching experience I've ever had. Makes the pain of preparing a flipped class feel like it's totally worth it.

Dealing with review papers

We've been studying the properties of galaxies, and the relationship between the distributions of stars (and dark matter) and the motions of those stars for a couple of weeks now. I assigned a pre-reading of half of a very good review paper (the Physical Properties and Environments of Nearby Galaxies) discussing this issue, knowing that this was a challenging pre-read (long, lots of information) but it is really good stuff, so I went with it anyway.

All students took longer than I had hoped for the pre-reading, and some of them took a factor of 5 longer than I hoped to read it. We talked about strategies for reading papers, and this is what they came up with (in no profound order):

• Lots of practice helps in reading papers, getting used to the format, jargon, parts you can skip, etc.
• Print out the paper, it's a lot easier than having it on the screen (and you can take notes/highlight)
• Get an idea of the overview of the paper via abstract and section headings
• figures are usually important
• Skip technical details/data, etc; you can pick it up later
• know what you're looking for in advance (it can even allow you to search for keywords, etc)
• make marginal notes

For non-native English speakers, we discussed also the importance of regular reading of non-technical literature and getting up to speed with jargon by reading simpler background material first.

Remote teaching

I had a trip to California last week for a workshop that I really had to go to. Knowing how hard it is to reschedule a class in a way that isn't unfair to at least someone, I decided to try an experiment - remote teaching.

We, and where I visited, have Polycom systems. So the thought was to use the polycom, me teaching from the conference room in California (which had a whiteboard, etc), and the students back in Michigan in our small conference room. We did two classes this way, both quite discussion orientated.

Well... it actually worked pretty well. Feedback was overall positive (I am likely to have to travel again for one class, and in a vote 8/10 students preferred us trying to polycom for the class I'd miss owing to travel rather than trying to reschedule). The interaction with the students was definitely less rich, and I was a lot less connected with the 'vibe' of the room, and couldn't listen easily into conversations to find out if things were on the right track or if there were problems/misconceptions, etc. But because the Polycom has such good sound and picture quality, and can zoom/pan, I was able to facilitate on the whiteboard in California, and the students could see that record reasonably clearly. I don't know if I'd want to be the only facilitator of a class without using something as reliable and clear as Polycom though.

Now, would I try this with an undergrad intro class? I don't think so - not unless there was a great 'hook' for me having to present from the remote location (e.g., I was at a telescope, and I'd have the class doing a tour of the telescope with me, for example) and there was someone in the classroom to facilitate there. But, as an option for more advanced classes, it might be worth considering.

What are my expectations?

I thought yesterday's class session would be a good time to check on the amount of pre-class work that the students put in to the pre-readings and pre-class homework, and to check on what kinds of notes they were taking (or planning to take) in class and what kind of processing they were expecting to do after class.

Now, these are graduate students, so they spent between 1.5 and 6(!) times more time than I expected on the pre-class work, with an average of about a factor of 4 more time than I thought. After significant amounts of discussion when we didn't quite talk the same language, it became clear that they didn't know what my expectations were for what/how much they should know.

Before term, I spent a lot of time (between 1.5 and 6 times more time than I should have?) on trying to be explicit with learning outcomes, and I thought being clear about what I wanted the students to be able to, and to know, on finishing this class. *And* I put sample oral exam final questions on the learning outcomes too, to give a sense of the kind of question that I wanted the students to be able to answer well. So all I needed to do was refer to that, right?

Very wrong. I realized that in a quest to make the learning outcomes and sample questions "rich", they were too vague and open-ended, and could be interpreted in almost any way; there were no reliable signals to the students about the level of knowledge/understanding/skills I wanted them to develop. Oops.

What was interesting after that was our collective struggle to figure out a way out of this mess. We had a jigsaw-format discussion (individuals listing the properties of spirals/ellipticals separately, separate spiral/elliptical groups discussing the properties of those galaxies separately, then splitting into spiral/elliptical mixed pairs to compare and contrast the properties of spirals and ellipticals, then a whole-class summary of that activity), where I tried to convey that the pre-reading is meant to give them the preparation they need for that kind of activity, but shouldn't be more in-depth than that. We also decided that they'd send their notes (or a summary of yesterday's main points) to me as part of the pre-class work for tomorrow. So we have a plan, at least...

What I'm curious about is if anyone has experience of how to communicate clearly and early performance expectations without it turning into something ridiculously fine-grained and micro-scale.

First day

I'm planning on doing this blog as a way for me to reflect as I start applying what I've learned about 'learner-centered' instructional techniques and methods.

I've never really been trained in teaching formally, but the Center for Research on Learning and Teaching at the University of Michigan has some awesome courses and workshops that inspired me to look into learner-centered teaching.

Today was the first day of the graduate galaxies class that I'm teaching this semester, and the goal of the class was to start practicing the skills of order of magnitude estimation (using this awesome resource). A big part of learning about galaxies (and order of magnitude) is having intuition about the scales that are typical of galaxies, how clustered they are, how much they vary in size, how dark matter is so much more extended than the visible component of galaxies, etc. So...  the students built a scale model (the scale is Milky Way = chocolate M+M). The image is one of the 5 group products (we also made the Sculptor Group, Cen A, Virgo [a lot of M+Ms died for that model] and a 'cosmic average' model [how many galaxies should be in that room]). My impression is that the activity went pretty well - I certainly learned a lot about relative scales, etc. and uncovered a couple of pieces of information about the Universe that I'm still a little surprised by (boy, galaxies are clustered!)

On reflection, I think that the activity would have gone better if I had included more specific preparation for this activity, and I talked too much during class; my goal for next time is substantially less talking, more listening and discussion-building.