New Eye on the Universe

Chile’s Vera C. Rubin Observatory and AI

Rubin Telescope.

The largest digital camera made to date, comprising 3,500 megapixels, is embarking on a decade-long time-lapse movie of the entire southern sky. The Vera C. Rubin Observatory, situated on Cerro Pachón, on the southern edge of Chile’s Atacama Desert,  has just made the transition from construction to full operations.  The project’s torrential data stream (20 terabytes per night) will enable searches for potentially hazardous asteroids, map out the detailed structure of our own Milky Way Galaxy and address some of the most profound open questions in contemporary physics. I first became involved in the Rubin Observatory a quarter of a century ago, and have watched its trajectory as it embarks on capturing an astronomical video of the entire southern sky.

While Covid-19 certainly had an impact on Rubin construction, an even more disruptive unanticipated change is currently under way. Generative AI (Chat GPT and its counterparts) is rapidly changing how we do science. I don’t think people are going to use the Rubin data in the ways we first envisioned, with database queries and rather traditional analysis methods. It seems increasingly likely that natural-language tools will empower sophisticated users to pose questions using prompts like “find me all the instances where a supernova exploded in a galaxy that is well-separated from its neighbors” and “is there a correlation between asteroid colors and their orbits?”  But before we get to the subject of AI, let me tell you a bit about the history of the observatory and my own personal involvement.

I’ve observed three very special things about the Rubin Observatory:

1) The 8.4 meter (27.5 foot) diameter of the light-collecting mirror allows us to see faint sources deep into the cosmos,

2) the wide-field optical system, nearly 10 square degrees, allows Rubin to scan the entire accessible sky in two nights of observations, and

3) the data processing system transfers images to computing centers to find anything that moves or changes brightness within minutes of the pictures being taken.  The entire archive of collected images will also be added together to obtain the deepest view ever collected of the entire southern sky.

The Vera C. Rubin Observatory in morning twilight, showcasing the modernistic structure that encloses the telescope, on Cerro Pachón. Note the vehicles, for scale, at the bottom of the image. The Rubin Observatory held its ‘”first light” celebration in June 2025 and is now beginning full science operations. Photo by Christopher Stubbs.

A full-size replica of the mosaic of 189 individual detectors that pave the Rubin focal plane. An image of the moon gives a sense of the wide field of view of the system. The combination of wide field and agile telescope lets the system capture successive 30 second exposures fast enough to scan the entire Southern sky in a couple of nights of observations. Photo credit: RubinObs/NOIRLab/SLAC/NSF/DOE/AURA).

A closeup of some of the first Rubin images shows colliding galaxies whose tidal gravitational fields have ripped out streams of stars. The smaller, redder, smudges are galaxies and clusters of galaxies at high redshift, i.e. large distances. (RubinObs/NOIRLab/SLAC/NSF/DOE/AURA).

The Rubin Observatory is not the only inhabitant of the high Atacama Desert. Photo by Christopher Stubbs.

On a crisp fall day in 2003, my first autumn at Harvard,four of us gathered in my newly furnished office to draft the first proposal for funding the detailed design of this ambitious new telescope. Now, nearly a quarter of a century later, the Vera C. Rubin Observatory has finished construction. I’ve been deeply engaged in the Rubin project over the entire course of its development, including serving as the inaugural Project Scientist, helping to refine the optical design, testing prototype detectors in our lab at Harvard, devising the calibration methodology for the project, overseeing the assembly of modules of the camera system, and spending much of my 2024-1025 sabbatical year in Chile working to help commission the system and bring it into full operation.

 The components that make up the Rubin system stretch the limit of state-of-the-art in many respects. The big light-collecting mirror was fabricated at the University of Arizona’s mirror lab, located below the University’s football stadium in Tucson. The maximum diameter of the mirrors they can make is set by the spacing between the vertical support columns that hold up the bleachers in the stadium seating! The 189 individual photon detectors that form a mosaic tiling across the focal plane stressed the world’s manufacturing capability, and it took many years to fabricate the quantity we needed. The camera, assembled at the SLAC laboratory in California, incorporates the largest precision optical lens ever made, and required an assembly tolerance of one half of a thousandth of an inch.

 It’s been an interesting professional transition for me to go from working in small groups of a dozen or so fellow scientists to participating in a project involving many hundreds of people, spanning three decades, with a construction budget of roughly $600 million. This sheer scale mandates a formal project management structure and a careful level of coordination. As the intellectual frontier in science and technology moves forward, problems become more complex, the measurements become more challenging, and the requisite mix of skills broadens. This has led in most scientific subfields to a trend towards increasingly larger team sizes and growing project complexity.  The doctoral preparation and training we receive in astrophysics does not typically include explicit training on how to successfully contribute to a project on this scale. After countless discussions with junior colleagues who were navigating how to best thrive in such a large enterprise, I tried to capture that advice in a book:  Going Big- A Scientist’s Guide to Large Projects and Collaborations(MIT Press, 2024). Whether this attempt at larger-scale mentoring is a success remains to be seen.

 The Chilean home for the Rubin Observatory was chosen after an extensive site selection process that considered sky coverage, the fraction of cloudless nights, manmade light pollution, existing infrastructure and atmospheric turbulence. Due to the proximity of the Andes mountains to the coast and the prevailing wind direction from the west, the smooth ocean-to-mountain airflow over Chilean observatories suffers less atmospheric turbulence than inland sites elsewhere.  As a result, starlight suffers less “twinkling” and we get crisp images of stars, galaxies and asteroids. The clear dry air of the Atacama desert offers an unobscured view of the sky above.

 There is something romantic about working as an astronomer in Chile. Due to the considerations described above, telescopes don’t tend to get built in unattractive places.  Getting to the Rubin Observatory from Cambridge entails first flying to Chile’s capital city of Santiago, then taking a one-hour domestic flight back northwards to coastal town of La Serena, the location of the Rubin project’s main offices and the Chilean data center. The construction team works the day shift, and the astronomers the night shift. Each day the bus to the observatory leaves the La Serena office compound at 6:30 in the morning, for the two-hour long drive eastward and inland, winding up the Elqui river valley towards the backbone of the Andes and the Rubin site.

 The floor of the Elqui valley contains countless vineyards that produce a varietal of grape used to make pisco.  This is the key ingredient in “pisco sour,” the Chilean national cocktail of choice. The road passes through a tunnel, adjacent to the irrigation dam that was completed in 2000. The width of the tunnel had to be made wide enough to allow the large astronomical mirrors to be transported uphill.

The large Rubin mirror being squeezed through the Puclaro tunnel, which was widened enough to allow passage of astronomical mirrors to the observatories up-valley. Photo credit: RubinObs/NOIRLab/SLAC/NSF/DOE/AURA.

The upper end of the Elqui valley, near the border with Argentina, has hand-crafted stone footpaths built into the hillsides, that date back to pre-Inca times. The portions that have survived centuries of earthquake-triggered rockfall remain visible today. Also up-valley are a number of meditation and wellness retreats, and observatories that cater to astro-tourists. Rumors of UFOs abound. Consequently the Elqui valley has an overall positive vibe that somewhat akin to a combination of the Napa Valley in California, and Roswell in New Mexico.  

 About an hour later, the bus takes the turnoff from Ruta 41, “Ruta de las Estrellas,” and enters the fenced-off reservation that provides a buffer zone for light pollution around the observatories. The uphill climb begins, culminating in the Cerro Pachón ridge where the Rubin telescope resides.  Upon arrival at 8:30 a.m., we stumble out of the bus, take a look overhead to see if we’re lucky enough to spot a condor,  and head towards the coffee machine. The team members who will work the night shift, on the other hand, remain decaffeinated and head to the dormitory for a nap.

 So who was Vera Rubin and why does this observatory carry her name? To answer that, we need to take a slight detour into some of the most profound open issues in modern physics. We might think that galaxies are mainly made up of the stars and gas we can see. Not so! Rubin was among the pioneers who showed, using observations of the internal motions of galaxies to measure the gravitational forces at work, that galaxies like our own Milky Way are about ten times more massive than the sum of their constituent stars. We have good reasons to believe that the “dark matter” that makes up most of the content of galaxies is something that lies beyond the periodic table of the elements, probably some as-yet undiscovered elementary particle.  

 And if that isn’t troubling enough, let me tell you about “dark energy.” In 1998, a team of us mapped out the history of cosmic expansion, with the goal of measuring the average mass density of the universe. We expected to see a slowing of the expansion, but were dumbfounded to learn that not only is the universe expanding, it’s doing so an ever-increasing rate. Evidently empty space exerts some kind of gravitational repulsion that drives the creation of even more empty space between galaxies. Stephen Weinberg, a renowned Nobel-prize winning physicist, called this discovery of the dark energy“a bone in the throat of theoretical physics.”

 Taken together, the dark matter and dark energy problems indicate a profound shortcoming in our understanding of fundamental physics, and the evidence for this comes from astronomical observations. The Rubin Observatory has been engineered to be a superb instrument to address these open questions. We’ll use a diversity of methods that exploit Rubin’s combination of depth, sky coverage and time-lapse images to better understand both dark matter and dark energy.

 That’s where AI comes in. Generative AI has already had a huge positive impact on my own research effectiveness. For about the past six years at the start of each summer, I’d say “OK, this is the year I’m going to teach myself how to program in Python.” The Python computer language totally dominates modern astronomical computing. The graduate students and postdoctoral researchers that surround me are all adept at it. Although I’m rather proficient in other computer languages, I repeatedly failed to make the transition to Python. That made it hard for me to exploit and contribute to the Rubin project’s software, somewhat to my frustration and that of my colleagues on the project.  

 With access to generative AI however, which is really good at writing computer programs, I’m now able to convert good ideas into functional Python code. There are usually errors that need to be identified and corrected, but that’s an unavoidable part of the process for anyone who writes complex programs.  The combination of my decades of experience and troubleshooting and these new tools have given me about a tenfold boost in productivity. Things I prototyped are now being incorporated into the mainstream Rubin operations and analysis pipelines.

 We’re now scrambling to find the best ways to meld together the strengths of rapidly evolving AI tools with the nightly operations and the data stream of the Rubin observatory.  I have every expectation that in the years ahead we’ll make increasingly good use of these new capabilities, with a corresponding boost to the ways Rubin, from its perch on a Chilean mountaintop, will enhance our understanding of the universe.

Author, Christopher Stubbs, in front of The Vera C. Rubin Observatory.