A collaborator sent over the code and said"just run it and you'll get results". I ran it for two hours and got four errors. In the end, I broke it into cells one by one before I found where the problem was.
My approach now is: when I get someone else's code, first break it into cells, then locate the issue cell by cell. I run it in the MuchSci science toolkit's interactive data analysis notebook.
After splitting into cells, the errors fall into four categories
Category 1: Missing packages in the environment. The first cell does only one thing: take the libraries you need and import them once to see which one turns red. Install whatever is missing, and see it all at once instead of finding out halfway through.
Category 2: Paths don't work. The second cell reads in the data and immediately prints the number of rows and columns and the first few rows. In someone else's code, the path points to a location on their own computer, so I have to change this step.
Category 3: Column names don't match. The third cell checks column names: print the actual column names in the data and compare them with the column names used in the code.
This category is the hardest to track down, because if a column name is off by one letter, some libraries don't raise an error and only silently give the wrong result. My habit now is: for any computed result, look at one row of raw values first before moving on. This is where the interactive data analysis notebook shines—the result of each cell stays in the notebook, so I can look back cell by cell to see which step first went wrong.
Category 4: Version differences. Some functions have different default parameters in different versions. I write out the parameters of key functions explicitly instead of using defaults.
After checking all four categories, I turn the code into my own
I don't use someone else's code directly; I rearrange it into four sections in the interactive data analysis notebook:
- Read in(the original file is left untouched)
- Clean(each action in its own cell)
- Compute(parameters written in the cell)
- Plot and export(figures and values come out together)
After rearranging, this code becomes my own analysis workflow. Next time I switch to a new batch of data, I only change the read-in cell.
I use this cell-splitting method in three situations
- Taking over someone else's data: first split into cells and get it running, then talk about conclusions.
- Reproducing results from a paper: rearrange the four sections according to the paper's methods section; if something doesn't match, find the difference cell by cell.
- My own analysis needs to be checked by someone else: hand over the whole notebook directly, and the other person doesn't need me to explain the intermediate steps.
The three supporting features
- The workflows and results in the notebook are archived into the MuchSci personal knowledge base, and I can retrieve them directly when writing the methods section.
- For the literature to cite in the methods section, use the MuchSci paper search assistant to verify DOI and citation information.
- For original methods descriptions in a foreign language, read them alongside the MuchSci paper translation.
- For the final written documentation, use the MuchSci office assistant agent to format it into an editable file.
From"code someone else gave me"to"my own workflow", it all comes down to this cell-splitting method. TheMuchSci interactive data analysis notebook keeps the audit trail,MuchSci personal knowledge base handles archiving,MuchSci paper search assistant,MuchSci paper translation, and MuchSci office assistant agent handle the final wrap-up.
Errors aren't scary; what's scary is silently getting the result wrong. This is also my first principle when using a graduate AI assistantfor data work: every step must be reviewable.