The project is organized around a series of sprints. Each sprint is approximately one month in duration, which is longer than recommended by many agile methods but much more appropriate given the (relatively) limited time available for a class project. The sprints are designed to provide experience with agile software development practices and, as such, have the following deliverables:

Each deliverable is described in more detail in the following sections.

Specification

[…] on any non-trivial project (more than about 1 week of coding or more than one programmer), if you don’t have a spec, you will always spend more time and create lower quality code. ~ Joel Spolsky

A specification formally defines what you expect to complete in each sprint. That is, the specification is an efficient way to communicate your work with others and ensure that you have consensus on the design before implementation begins in earnest.

Specifications generally cover the following information:

Overview
At a high level, what are you trying to do? Why is it important?
Problem
Describe the problem in greater detail. What are the functional and non-functional requirements of the solution? It is often very helpful to include a use case that describes the issue from a user’s perspective.
Design
What specifically do you intend to do? Here is where you include technical details: someone else who reads your specification and is familiar with the system ought to be able to implement the design without any additional information. It’s also helpful to identify alternatives for consideration, both to show that you’ve thought about them and in case your initial plan doesn’t pan out as you intend.
Work Items
Who is responsible for the work? What is the breakdown of tasks?
Testing
How do you intend to show that you’ve addressed the problem?
Documentation
What updates must be made to the system’s documentation so that others benefit from the work? Source code comments are generally expected, but user-facing functionality requires documentation in other formats for the new capabilities to be useful.

It is recommended that you complete your specification early in the sprint so that you have time to solicit feedback and allow time to (possibly) change its design and implementation.

The specification template is derived from those used by OpenStack. If struggling to understand what amount of detail to include in the specification, one approach is to review a a few OpenStack specifications to get a feel for what is required to manage a large open source project. Some specific examples that I oversaw while contributing to OpenStack follow:

If you skim these examples, you’ll see that the amount of detail varies depending on the complexity of the change(s). For example, the specification for ephemeral storage encryption is very high-level because of prior work on similar features whereas certificate validation is extremely detailed. In general, a more detailed specification makes the implementation easier to review and results in more rapid approval of the corresponding code changes.

A more entertaining resource is Joel Spolsky’s series “Painless Functional Specifications,” which addresses this topic humorously and pragmatically.

Implementation

Project artifacts reside in a Git repository, which provides permanent record of all changes. The basic workflow is

  1. clone the repository,
  2. create a branch,
  3. make changes (i.e., commits),
  4. push your branch to the remote repository, and
  5. create a pull request for your changes.

If you aren’t comfortable with Git, then Pro Git is an excellent resource that not only walks through its basic use but also goes into considerable depth on its internals.

When committing anything to the repository, remember that each “logical change” should appear in its own commit. A logical change may be a complete specification, the cleanup of existing source code (e.g., reformatting it to follow an agreed-upon coding standard), or the addition of a feature. In general, the following should be avoided:

Mixing functional and non-functional changes
Create separate commits for the non-functional change (e.g., code cleanup) and the functional change; starting with the former ought to simplify the latter.
Mixing two unrelated functional changes
Break the changes apart into separate commits.
Having a giant commit for a feature
Start with the minimum functionality that provides business value (i.e., value to the end user) and add capabilities incrementally.
Mingling auto-generated with hand-written code
Often a gotcha when using web frameworks, clearly delineate the project initialization (e.g., the scaffolding provided by the framework) from its project-specific modifications to aid with review.

Typically this means that each commit has fewer than 200 lines of changes. Why is this important? Larger commits are more difficult to review, increasing the likelihood of mistakes. In addition, knowing the context for the change (e.g., who changed the line and why it was modified) is often critical to maintain the system’s functionality during later refactoring.

All changes (specifications, source code, documentation, etc.) must be reviewed prior to acceptance. Reviews ensure that others understand and agree with the proposed changes – another set of eyes often finds mistakes that the original author(s) missed!

Pull Requests

A pull request is a solicitation for a project’s maintainer to accept a change from another contributor. In some cases, the pull request is accepted immediately, without any modifications; more commonly, though, the project’s maintainer reviews the proposed change and requests modification prior to accepting (and merging!) the pull request.

Best Practices

Use lots of pull requests! In particular, logically separate changes should appear in separate pull requests. Examples of logically separate changes include commenting existing code, implementing a new capability, and adding feature specification.

Disentangling logically-separate changes from a series of commits can be challenging. Ask if you have questions about this process!

Takeaway: Create a separate pull request for every change that you want to make.

Pull requests typically receive comments that should be addressed prior to being accepted. It’s unlikely to have additional changes accepted when they build on existing changes. For example, a 500-line pull request that builds on an open 200-line pull request won’t be approved until after the smaller one. Consequently, receiving approval for any open pull requests should take immediate priority.

As you think about implementing new features, focus on the minimum viable product. That is, what must be added before a typical user could take advantage of your work? Better to complete a small capability than have something that is unreliable or even unusable.

Takeaway: Try to close outstanding pull requests as quickly as possible. Aim to complete a minimum viable product by the conclusion of this sprint.

Retrospective

At the end of each sprint, your team should hold a retrospective to discuss the good, the bad, and the ugly. The retrospective is an opportunity for reflection. Think about the issues that your team encountered during the development cycle, consider their cause(s), and discuss possible resolutions. The retrospective is not the place for finger-pointing; rather, it is an invitation to explore how to improve the process so that the team is more successful the next iteration. (If individuals aren’t contributing equally to the project, the cause may be related to the rest of the team or the process, and soliciting their perspective is critical to resolve the underlying issues.) Also be sure to celebrate the team’s achievements and what went well. There may even be additional opportunities for improvement in these areas.

You’ll probably benefit from appointing a scribe to take notes during the retrospective. Organize these notes into short synopsis (1/2 – 1 page) of the sprint as a whole and what would make the team more effective in the future.

A major goal of a retrospective is to enumerate issues and lessons learned. To do so effectively requires being specific. For example, it’s great to record that you started working too late in the sprint, but how late was “too late”? Maybe it was the night before or possibly the week before the end of the sprint. While that information is obvious to you now, it won’t be in a month or two. Stated differently, there should be sufficient detail to prevent another team from experiencing the same issue in the future.

Likewise, be specific when describing changes for future sprints. It’s difficult to measure improvement without having a specific objective. Using the prior example, stating that you will start working earlier in the sprint is great, but what does that mean practically? An hour earlier technically satisfies this objective but practically will make little difference. The key is realistic objective setting where satisfying those objectives will improve your effectiveness as a development team. There’s also no need to sugarcoat the lessons learned: if starting the night before was a bad idea, then admit it and move forward. That admission, by itself, will not decrease your grade.

Takeaway: Be specific in the sprint retrospective.

Reviews and Iterative Development

Probably the best real-life software engineering lesson is the amount of scrutiny applied to proposed changes to production software. There’s excellent rationale for doing so: In many cases, the original author(s) of the code will not be those who maintain it, and consequently, the code must be not only functional but also understandable and extensible.

That perspective is very different from traditional course assignments where students submit whatever they’ve completed when the due date arrives (and the maintainability of the code takes a backseat to its functionality). For this project, though, reviews of proposed changes (in the form of pull requests) mimics the experience of contributing to open source projects.

Success requires teams to address my feedback fairly quickly, and the more that you address, the higher your grade. That is, this feedback is essentially a free opportunity for teams to improve their deliverables. Of course, when code isn’t available for review until just before a deadline, there isn’t an opportunity to provide such feedback. Consequently, many of the proposed changes require substantial revisions before they can be “approved” by a project maintainer. In the real world, that means that they aren’t going to be part of the next release (because they aren’t yet ready).

Takeaway: Use the iterative development cycle to your advantage. Address feedback on pull requests as quickly as possible. To receive full credit when grading the sprint deliverables, your proposed change should be approved and merged prior to the end of the sprint.

Resources

How to do a Sprint (Teddy Drewes)