CIS 3500

Introduction to Software Engineering
or how to JOYFULLY BUILD with a TEAM

Prof. Jérémie Lumbroso, Ph.D.

lumbroso@seas.upenn.edu

LECTURE 4:
Code Archeology

⚠️➡️ RIGHT NOW, go to https://sli.do

Code #23232323 

"Programming Archeology"

A scifi author imagines how a civilization in a thousand years might read our code written today

• Unix Epoch (start): 00:00:00 UTC on January 1, 1970

• Purpose: Acts as the reference point for time in Unix-like systems

• Measurement: Time stored as the number of seconds elapsed since this epoch

• Moon Landing: In July 20, 1969, Neil Armstrong and Buzz Aldrin are the first men to walk the moon (Apollo 11 mission)

Vernor Vinge

C.S. professor (died March 2024), author of many sci-fi novels, he popularized the notion of technological singularity.

• The earliest ideas resembling the singularity date back to John von Neumann in the 1950s.
  • Stanisław Ulam, in 1958, recalled von Neumann discussing how technological progress was accelerating toward something incomprehensible.
• I. J. Good (1965) introduced the concept of an “intelligence explosion,” suggesting that a sufficiently advanced AI could recursively improve itself, leading to superintelligence.

• In 1983, Vinge explicitly used the term "singularity" in his essay "First Word" (Omni magazine).
• His 1993 essay, "The Coming Technological Singularity," outlined the idea in a structured way, connecting AI, intelligence explosion, and runaway progress.

HW1

Diving Into A Codebase

HW1 Content

• You are introduced to a static website generator called Hugo
  • This is the several generations after WordPress
  • Similar tools: Jekyll, Pelican, Gatsby, etc.
• Static website generators create websites from content (in Markdown, a simplified document language) and themes (predefined)
• You will first learn by doing 2-3 tutorial starter exercises, including a Hello World to understand how Hugo works
• The main exercise is modifying an existing landing page

HW1 Hugo Theme's Example Site

Original

Student work

Student work

Student work

Student work

Student work

Sample HW1 From Spring 2024

Homework 1 – The "Tricycle" Version of Coding

• Goal: Use Hugo (a static site generator) and an existing theme to create a fake landing page for a fake product.
• This is a form of Code Archaeology—you're parachuting into an existing, somewhat unknown codebase (the Hugo theme).
• You'll practice:
  1. Looking up the technology (Hugo basics).
  2. Doing easy tutorials or the official docs to see how everything works.
  3. Building up from an example or starter (the theme's exampleSite).
  4. Troubleshooting typical problems in static site compilation and theme usage.

Code Archaeology: Case Study Building A Hugo Website

"Programming Archaeology is about diving into an existing codebase—where everything's in place, but you don't know exactly how."

What's Special About Hugo and a Static Site?

• It doesn't "run" like a normal app—instead it compiles Markdown + templates into static HTML/CSS/JS.
• The theme (template) you use might require certain Node.js dependencies or an SCSS pipeline.
• You can still break things: Typos in template partials, missing front-matter, or failing to install required dependencies can cause "compilation errors."

Why Hugo (and Why Now)?

• Safe environment: "Tricycle version" of taking on unknown code. It's simpler than a large, multi-language codebase (like a big monolith in Java/Python).
• No complicated runtime: The worst that happens is a failed build or a broken layout. No real "segfaults" or "service outages."
• Fast feedback loop: hugo server can live-reload changes—perfect for exploring code step by step.
• Realistic scenario: In the real world, new devs often join an existing project. You jump into a codebase where architecture isn't always clear.

Don't Worry!

• Although we are starting with a static website generator, we will examine more difficult projects
• You will explore a Chrome extension from past semesters and learn how to make changes
• You will also do a code analysis of a Hackathon project
• You will also write tests for an existing codebase
• ... but let's start small!

Learning Goals

1. Develop a Working Model: Figure out how Hugo organizes content vs. layouts.
2. Configure an IDE or Editor to help navigate the code quickly (search across files, syntax highlighting for Hugo templates, etc.).
3. Static & Dynamic Strategies:
   • Static: Searching for where certain CSS classes or partials are defined.
   • "Dynamic": Using hugo server to see live results of your changes.
4. Troubleshooting: Identify typical pitfalls (missing dependencies, front-matter errors, node-sass issues, etc.).

Key Archaeological Principles

When diving into any codebase, especially for the first time, keep these principles in mind:

1. You Won’t Understand It All

   • Accept upfront that no one truly understands every detail of a non-trivial codebase. Aim to build a mental map of the parts you do need to change or configure.

2. Be Systematic

   • Don’t randomly click around or guess. Start with documentation (official or in the project’s README.md), then systematically investigate files and directories.

3. Work Iteratively

   • Make one change at a time, see what happens. If something breaks, revert and re-check. This approach keeps you from being overwhelmed.

4. Document as You Go

   • Take notes about what “partials” do, how _index.md differs from a typical content file, or how front matter influences build behavior.
   • Write down paths and filenames. It’s your personal “archaeologist’s field journal.”

How to Tackle This Hugo Codebase

We will see that the principle here apply to any codebase.

1. Leverage Previous Experience

• If you've done front-end dev or theming before, your instincts help (HTML/CSS, placeholders, partials).

2. Consult Documentation

• Hugo's docs: https://gohugo.io
• The "Building With Hugo" book chapter.
• Your theme's README and exampleSite.

3. Use an IDE / Search Tools

• Tools like Visual Studio Code or grep or ag to find relevant pieces.
• Example: "Where is the string ‘Hugo Bootstrap Theme' in the theme?" => can lead you to the partial for the features section.

4. Trial & Error

• hugo server -D to see draft posts, watch for errors.
• Modify your content or layout in small steps, see if it breaks or compiles.

The "Big Pile of Code" – Hugo Theme Edition

• A Hugo website typicall includes:
  • config.toml or .yaml with site configs.
  • themes/hugo-bootstrap-theme/ (or whichever theme you chose).
  • Example content files, partials, SCSS, JS, images.
• The code structure can feel overwhelming at first: "Where does the real layout come from?!"

Key Tip: Try to see how "content" maps to "layouts" by looking at the theme's partials and searching for references to them in your _index.md or config.toml.

"You Will Never Understand the Entire System!"

• This is especially true if you peek into themes/hugo-bootstrap-theme and see hundreds of SCSS, partials, shortcodes, etc.
• Your goal is not to fully master the entire theme. Instead:
  1. Focus on the entry points relevant to your landing page.
  2. Find the partials or pages that produce the "hero banner," "features list," "footer," etc.

Software is Massively Redundant

• Often you can find a snippet of code or front matter in the exampleSite directory.
• Copy/paste as a starting point—just rename or tweak images, text, etc.

"Code Must Run to Do Stuff"

• In a static-site world, "run" means "compile or build successfully."
• If something breaks, you get a build error in the console (like a snippet of Go template syntax or missing variable).
• Keep an eye on your terminal output when you call hugo server.

Entry Points

• Hugo's main "entry" is the hugo command, which reads config.toml and scans content/.
• The theme's partials get called from the layout files.
• If you get lost in a partial, search the code to see who references it.

Code Must Exist (But Where?)

• Binaries or Node modules might be in node_modules/ if the theme uses them.
• Actual site code is in your content/ folder and layouts/.
• The theme's library code is in themes/hugo-bootstrap-theme/.

Probing Code Transparency

• Transparent (local) vs. Translucent (precompiled theme code) vs. Opaque (some minified CSS or JS).
• In your assignment, you mostly deal with transparent code because you have the theme source.

Strategy: How to Tackle Your Hugo Theme

1. Start with the Official Docs

   • Hugo has thorough documentation and an active community.
   • Checking the docs can clarify how front matter or partials work, so you know what to search for in the code.

2. Peek at the Project’s README.md

   • Some themes or code samples have mini guides. They might detail how to compile assets or reference special partials.

3. Cloning, Building, and Running

   • git clone <the-hugo-theme> and see if there’s a package.json that requires npm install.
   • Try hugo server or npm run dev—does everything build? If not, read any build errors carefully.
   • Fix environment issues: Do you have the right Node.js version? The extended version of Hugo?

4. Look for Where to Insert Content

   • Usually, _index.md drives the homepage content.
   • Layouts might be in themes/theme-name/layouts/. The _default/ folder often has baseof.html, single.html, list.html.
   • Hugo partials (like header.html, footer.html) are in partials/.

5. Use Your IDE and Text Search

   • Searching for a relevant keyword (like nav, header, hero) across the code helps locate the partial that controls the top banner.
   • Searching for the string “title” may help you see how the site title is injected. This is especially useful if you suspect a naming or config discrepancy.

6. Make Small Changes

   • Add a console log in the JS, or an extra <p> in a partial to confirm you’ve found the correct place.
   • Rebuild the site. If something breaks, revert quickly or review your commits.

7. Review the “Front Matter” Requirements

   • Hugo uses front matter to define data (e.g., draft = true, title, tags).
   • If you need new fields (like landingPagePromoImage), you can add them to front matter and reference them in your layouts.

8. Document Findings

   • You may want to keep a short wiki or notes doc: “To change the hero banner, edit themes/mytheme/layouts/partials/hero.html.”
   • This practice cements your learning and helps if you or your teammates revisit the code later.

Steps to Understand the New Hugo Codebase

1. Look at the README (or theme docs).
2. Clone or copy the exampleSite config into your root to see working defaults.
3. Build & run with hugo server; confirm no errors.
4. Search for text or images you want to replace.
5. Edit content or partials in small increments, checking the browser each time.

IDE & Code Navigation

• Recommended: VS Code with the Hugo extension.
• Search Across Files:
  • In VS Code, use Ctrl+Shift+F (Windows) or Cmd+Shift+F (macOS) to find where a snippet occurs.
  • Exclude themes/* if you want to focus on your local site files first; or include it if you suspect the logic is in the theme.
• Explorer: Expand and collapse directories to see how partials, layouts, and content are organized.

Observation: Tools & Techniques

• Static Gathering:
  • Searching the theme directory for strings or partial names.
  • Reading the top-level config.
• Dynamic Gathering:
  • hugo server -v (verbose) if you need logs.
  • Reload the site in your browser, open DevTools for console errors, etc.

Software is Full of Patterns

• Hugo has conventions:
  • layouts/_default/single.html
  • layouts/partials/ for small reusable chunks
  • The archetypes/ folder for default front matter if you run hugo new commands

Observation: If your theme has a pattern, replicate it for new features (e.g., if it uses shortcodes for call-to-action banners, copy the existing pattern for your new "feature highlight").

Hugo Site Structure: A Quick Refresher

A typical Hugo project may look like this:

• config.toml: Site-wide configuration (e.g., site title, baseURL).
• content/: Markdown files that hold actual site text.
• layouts/ and themes/: The architectural heart. Houses templates, partials, and the scaffolding that stitches content to rendered pages.
• static/: Unprocessed assets (images, CSS, JS).

In your homework, you’ll have to figure out where to make changes so that the site’s final design (like your “fake landing page” for your “fake product”) matches specs.

Keeping everything in your mind is hard!

From Andrew Lindesay - Software & Data Engineering

Sample Diagram Explaining Hugo Rendering Workflow

Document & Share Findings

• Keep notes in your repository's README or a separate NOTES.md.
• Summarize what you learned about partials, what file controls the homepage, etc.
• This helps you (and future you) to remember how you set up or modified the theme.

Common Pitfalls & Observations

1. Theme Dependencies:

   • Some themes (including the one we use here) require precompilation
     • I.e., with Node.js, you to run npm install (for SCSS, PostCSS, etc.).
   • If you skip this, your site might have missing CSS or fail to build.

2. Base URL / config.toml:

   • A wrong baseURL can lead to broken images.
   • Double-check your site is accessible locally.

3. Front Matter:

   • Missing or malformed front matter in a Markdown file can cause blank pages or errors.
   • Check for the triple-dash lines (---) or correct parameters.

4. Search & Replace Mistakes:

   • Be cautious when globally renaming placeholders.
   • ALWAYS check commits individually.

5. Node-sass / SCSS Issues:

   • If your theme is old or pinned to a certain node-sass version, you might see compilation errors.

More Common Pain Points in Hugo

• Theme Asset Compilation

  • If your theme uses Node-based tooling (e.g., Webpack, Babel, Sass), you might see instructions like npm run build or gulp build.
  • A typical error: “cannot find module” or “invalid command” because you forgot to npm install.

• Partial Overriding

  • If your custom partial or layout shares the same name/path as the theme’s partial, your local version will override the theme’s.
  • This can be helpful or confusing if you’re not aware you’re overriding a default partial.

• Content Not Showing

  • Possibly the front matter draft is set to true. Or your _index.md file is in the wrong folder (like content/index.md instead of content/_index.md).
  • Another possibility is that the “archetype” for your content type is not recognized, or you have a mismatch in folder naming.

• Permalink or URL Issues

  • You might see broken links if your baseURL in config.toml is incorrect.
  • Or you might have an old link from a template referencing an absolute path like http://localhost:1313/... when you want relative links.

Closing: Goals of HW1

1. Your job: Create a landing page using an unfamiliar theme.
2. Archaeology angle: Skim, search, hypothesize how the theme is structured, then experiment.
3. By the end, you'll have a functioning site. Next homework, you'll see how to deploy it automatically.

Tips for Future Code Archeology

1. You’ll Never Master Every File: That’s OK. Focus on the parts that matter for your immediate tasks.
2. Value of Good Documentation: When you tweak code, leave behind helpful comments or short docs (like a NOTES.md) for yourself and future students.
3. Learn to Investigate: Don’t be afraid of building from local docs, or looking at the official Hugo docs. Searching is fundamental.
4. Stay Curious: Always wonder, “Why is it done this way? Where is that variable coming from? Is it in the config or front matter?” This curiosity fosters deeper understanding.

The COBOL Story

A Real Case Study In Software Archeology

Software Archaeology & COBOL

• COBOL (1959): Among the oldest high-level languages still in use, primarily in finance, government, and administration.

• Why It Matters: Despite its age, COBOL underpins critical systems worldwide—but finding experts to maintain it is getting harder.

COBOL's Staying Power

• History & Design
  • Created in 1959, influenced by computing pioneer Grace Hopper.
  • Tailored for business operations (easy-to-read, data-centric).
• Widespread Adoption
  • Banking & Finance: Manages transactions at scale.
  • Government Agencies: Processes taxes, social security, and benefits.
  • Insurance: Policy management, claims processing.
• Why It’s Still Around
  • Proven reliability over decades.
  • Migration costs to newer tech are huge.
  • Regulatory requirements favor stable, well-tested systems.

COBOL systems are deeply ingrained in critical infrastructure and remain functional, leading many organizations to keep them rather than replace them.

The COBOL Programmer Shortage

• Demographic Shifts
  • Many experienced COBOL developers are retiring.
  • Few new programmers are choosing to learn COBOL.
• Educational Gaps
  • University curricula rarely include COBOL or mainframe topics.
  • Training mostly happens on-the-job or through specialized programs.
• Economic Implications
  • Limited talent drives up costs for maintenance and modernization.
  • Outages or slow updates can cause severe financial and reputational damage.

As COBOL experts retire, a looming skills gap endangers the stability of crucial applications running on legacy systems.

Unemployment System Crisis (2020)

• Context
  • Early in the COVID-19 pandemic, unemployment claims skyrocketed.
  • Many state unemployment systems (some built in COBOL decades ago) couldn’t handle the surge.
• Problems Faced
  • System Overload: Outdated COBOL mainframes buckled under unprecedented demands.
  • Payment Delays: People waited weeks or even months for benefits.
  • Public Outcry: Governments faced pressure to fix these failing systems quickly.
• Emergency Response
  • State governments issued pleas for retired COBOL programmers to come back.
  • Quick patches and scaling attempts were made to handle mounting claims.
• Lessons Learned
  • Dependency on Legacy Systems: Vital public services depend on old tech.
  • Need for Preparedness: Systems must be ready for sudden spikes in usage.
  • Importance of Skills Retention: A shortage of COBOL talent made crisis recovery harder and slower.

The 2020 unemployment surge highlighted how vulnerable essential services can be when they rely on decades-old, under-maintained COBOL systems.

Looking Ahead – Possible Solutions

• Training & Education
  • Corporate and university initiatives to teach COBOL basics.
  • Mentoring programs pairing new developers with retiring experts.
• Modernization Paths
  • Replatforming: Move COBOL apps to modern hardware or cloud environments without fully rewriting them.
  • Incremental Refactoring: Gradual transition of core components to newer languages or microservices.
  • API Wrapping: Expose COBOL functions via APIs, allowing modern front ends to interact with legacy back ends.
• Strategic Planning
  • Retaining retirees as consultants to transfer knowledge.
  • Documenting systems thoroughly to reduce reliance on individual expertise.

Organizations can either modernize legacy systems or invest in COBOL training—but doing nothing increases the risk of critical failures.

Conclusion

• COBOL’s persistence underscores how “old” doesn’t always mean “obsolete.”

• It highlights the importance of software archaeology—understanding and maintaining legacy code to ensure the continued stability of society’s most essential services.

THANK YOU

Questions?