What is Software Engineering?

Of course there are so many answers to that question from the simplistic to the outrageously formal. In this post I will give an answer that outlines SESCs view and concentrates on being pragmatic.

A Definition

This is not a hard and fast, text book definition its just one we can work with for now. Software Engineering is:

The use of processes and associated tools to write good quality software that produces excellent scientific results

The processes we are thinking of need not be tedious and long-winded but having a process or processes for different tasks will help organise your work and keep development under control. Using tools to help with the processes saves time and means it is easy to keep a consistent method of working for yourself and new developers. There are also tools to measure the quality of your software and help you build and test its correctness. Figure 1 shows the elements that are traditionally included in software engineering and these form a basis for our thinking. It looks like a waterfall with one stage leading neatly to the next but we know thats not how things happen in practice. Development is an iterative process with a variety of time scales and different level of detail required at each stage.

Elements in Software Engineering

Figure 1: The Usual Elements of the Software Engineering Process

In the next section Ill put up some brief ideas for each of the elements. My aim is to be pragmatic and you shouldnt read these as prescriptive and certainly not definitive. Read around to get other ideas.

Some Ideas

Requirements

  • Must be written down
  • Write them at the head of a file for a simple script (along with your name and a date)
  • Scripts could print out a usage message if run with no arguments - kind of requirements in miniature
  • Write a more formal document in Word, LaTeX, etc.
  • Keep them under revision control so you can see the latest version, previous changes and share them with others

Design

This is a hard one. Complete up front design doesnt work unless you can foresee every little problem, no-one is going to change the requirements (ha!), and you are draconian about the implementation. The hardest part is to keep the design in sync with the actual implementation. One approach is to call the design an initial design an note that it is subject to change. Then you can use tools to generate a detailed design from the code. Some high level diagrams can then be used in an updated design document.

  • Dont bother for small scripts - comments will be enough. You do use comments dont you?
  • But be prepared to start a design if the script becomes larger than one file
  • Go for high level first
  • Draw diagrams. I have used dia for some of my designs
  • If you have any ideas about lower levels write a brief description
  • Keep documents under revision control
  • Detailed design can be retrieved from the code by tools such as doxygen (see this article{no content}) and Understand

Implementation

There is no way Im going to say which language you should use1 nor anything about a development environment! But here are some useful ideas.

  • Use comments:
    • at the head of each file to say whats in it
    • for each function, subroutine, data structure, class, class method &
    • in your implementation.
    • make them useful - e.g. describing decisions you made, limitations of functions, relation to other parts of the code
  • There are tools that can extract comments with special mark up to produce documentation, e.g. doxygen
  • Keep files under revision control
  • For compiled langauges use a build system e.g. make to make compiling your code easier
  • Make use of the compiler to give you warnings about your source code
  • Use the compiler or another tool to check that your code conforms to language standards

Verification

By which I mean testing. Or verification and validation.

Verification is making sure the software meets the requirements. So having the requirements written down is necessary here. Then you can copy the requirements to a new document, and for each of them describe how to make the software do it. Theres a verification document.

Validation is making sure the software gives the right answer which is what the user really wants. You need test input data, the corresponding output data and a method to check the output whn you run the software now matches the output from before. This is often called regression testing by those who like to use the jargon.

You could also consider:

  • Unit testing - test each small piece by setting up the appropriate inputs and checking that it behaves as anticipated
  • Integration testing - do the functions, classes you have implemented work correctly as groups?
  • Coverage testing or analysis - determine how much of the code you wrote is actually executed by the collection of test cases you have.
  • Destructive testing - what would happen if a million monkies could use your software?
  • Performance testing - do individual parts and the code as a whole perform acceptably in terms of time to output, scalability, resource usage, for example?
  • Use a system to make running the tests simple. Your build system would do.

All of these need some sort of plan before you begin and a report when its all done. Keep these documents and any code to run tests under revision control.

Maintenance

  • Do your users know who to contact for help or to report bugs?
  • Can they ask for new features?
  • Do they know how to cite your software. How about CITATION files?
  • Use revision control to keep track of your source code. This is especially useful for finding bugs during development and working with users on old versions.

Conclusion - Think of Others

To judge what is appropriate software engineering for you always think of others. Others in this context includes your future self. What do they/you need to know:

  • to use the software
  • to trust the results they get
  • to be able to include it in their own software
  • to work with you in developing new features
  • to give you another grant to carry on the development

If youre working in a £million project with international collaborators over 10 years then you will need some pretty heavyweight processes and tools to support them. If its a smaller project with collaborators you know well and interact with often then something lightweight could be appropriate. If its a convenient piece of scripting for yourself then I guess something minimal. But beware, someone else might find it useful too and want to know how to use and extend it!

And if you didnt notice the repeated remarks about revision control then Keep your files under revision control. Ill post more about work flows with revision control in the future.

Here to Help

Software Engineering is coming to research software and SESC is here to help. If you cringe at the thought of software engineering, dont worry, SESC can help you get started. If you already use some software engineering tools or process, thats great, SESC is here to help you go further.

Feel free to contact us to get things started.

(1) Use Fortran