Systems Development for
Harvard John A. Paulson
School of Engineering
Computational Science
CS107/AC207
and Applied Sciences
Fall 2023
I. Becker Troncoso
Oce: PSEC 1.312-05
Pair-Programming 6
Pipeline: SQL, Python, Bash
Issued:
Due:
October 16, 2023
October 27, 2023 09:59pm
In this pair-programming session you will consider a simple implementation of a pipeline
and perform data analysis on the astronomical data.
You should work on the exercises in groups of 3 to 4 students via a tmate session. Your
team members can submit the same file. Please indicate your names in a header in the
files. See the tutorials on the class website for an example pair-programming workflow.1
Do not forget to commit and push your work when you are done. Ensure that you are on
your default branch for this and not, possibly, on your homework branch.
Exercise 1: Pipeline
Deliverables:
1. exercise_1.sh
2. 1a.py
3. 1b.py
4. 1c.py
5. 1d.py
6. 1e.py
The goal of this pair programming is to create a pipeline for astronomical data processing
utilizing a bash script. In this task, we encourage you to harness your proficiency in scripting to develop a system where input queries related to celestial phenomena are processed
using Python and SQL. After processing, seamlessly compile the results into a structured
text file.
Before you start, install the python library astroquery. You can read the documentation2
to learn how to use it. Note that the behavior is different than SqlLite3.
1 https://harvard-iacs.github.io/2023-CS107/pages/tutorials.html#tutorial-pp-tmate
2 https://astroquery.readthedocs.io/en/latest/gaia/gaia.html
1
Pipeline instructions: Create a bash script, which receives as input an object denomination. This is the name of the object, such as "pluto" or "eros". Pay attention since some "_"
on the first table are represented by " " on the second table, e.g. "2001_sc288". The script
should execute the following:
• Create the python script "1a.py" which generates the query to the table gaiadr3.sso_orbits
to get the orbital parameters inclination, eccentricity and semi_major_axis of the selected object. Save the query into "query1.txt".
• Create a python script called "1b.py", which reads and executes the query from
"query1.txt", and save the results into a temporal file "temp1.txt".
• Create the python script "1c.py" which generates the query to the table gaiadr3.sso_observation
to get the epoch, ra, dec and g_mag, only if astrometric_outcome_transit is one and
the selected object. The output should be save into "query2.txt".
• Create the python script "1d.py", which reads and executes the query from "query2.txt".
Save the results into a temporal file "temp2.txt".
• Create the python script "1e.py" which plots the time series of epoch vs g_mag. In
the subtitle the plot should contain the orbital parameters, if they exist. Save the
plot.
• Remove all the generated files, except the plot and the python scripts.
• Print "0" in the terminal.
2