Is this a test or specification?

February 26, 2008, No Comments

I wonder what you’d call this. Would you say this is a test? Or would you say this is specification? Please explain in the comments and/or take the poll.

Story: User creates account

	As a user
	I want to create an account
	So I can add feeds and create new ones

	Scenario: Successful signup

	Given a username 'jack'
	And a password 'test'
	And a password confirmation 'test'
	And an email 'jack@test.ch'

	When user wants to create a new account

	Then a new email is sent to 'jack@test.ch'
	And it contains an activation code
	And it contains username and password
	And user isn't able to log in

	When user clicks on activation code

	Then user is able to log in
	And receives an email with the sign-up notification

So what do you think: Is this a test? Is this a specification? Both? None?

Let me know what you think in the comments.

From idea to reality

November 11, 2007, No Comments

To be able to increase the value of our projects, this is the main thing we have to pay attention to: How quickly can we make a new idea a reality?

Reducing Cost Of Software Projects (’Why we get paid’-Series)

November 3, 2007, No Comments

a: cheaper people or smaller team

The most obvious way to keep cost small is to hire people that cost less. What is a lot less obvious is the negative impact this practice can have on both: b and c. While the per month cost is lower, the project is likely to get done later, and maintenance will be more difficult.

A more promising way is to use a smaller team. For example use 5 instead of 15 people. By cutting down communication overhead, b may not even be threatened.

b: be done earlier

Everybody would like to be done earlier. Having people work overtime isn’t the way to get there. Why? Because if you try it this way, you’ll likely increase a and c, resulting in bigger overall cost.

There are several things that can help you get done earlier:

• Simple systems (don’t use technology you don’t need)
• Yagni - You ain’t gonna need it (don’t create things you think might be necessary some time in the future)
• Get started early (don’t specify every detail of the system)
• Technology (good frameworks, programming language etc.)

c: maintainable code base

Creating maintainable code sometimes competes with a and b. Some of the things that help keeping maintenance cost down:

• Well designed code (another argument for good developers)
• Good communication (the more people know about the software, the better the chance that somebody knows what’s going on)
• Original team maintains code
• Automated tests

Good documentation is only a last resort.

What other practices do you see, that can reduce the overall cost of the project?

Why testers fear Agile Development

October 28, 2007, 1 Comments

Let’s assume (for the sake of this argument) that the programmers can build one small idea in the first week. Then a second small idea in the second week. They even have time enough to fix the few defects they introduced in the first week. Let’s say they can sustain that pace forever.

The program grows steadily. But what happens to testing?

Manually test each iteration

If we manually test each finished idea, the diagram above is what’s going to happen. That’s because we have to make sure all previously built ideas still work properly. Obviously it wouldn’t take long for testing to cost a lot more than programming. And it’s almost certain that at one point in the future, the testing will not be done, when new software is ready.

So that doesn’t work.

Manually test each release

Why not wait for a while, and then test a set of ideas together - so we only have to test them once. That’s what’s going to happen:

Because there is more and more software, tests will take longer and longer.

With all defects of 6 iterations found, the programmers won’t be able to work on a new small idea for a week. So the 12 ideas that could have been done in 12 weeks are now done in 13.

Progress is getting slower and slower. That’s not what Agile promises or is about.

No Manual Testing

That’s why automatic tests are crucial for Agile development. There is no way around it. If you want to make steady progress, you need automatic tests.

It’s the teams responsibility to make sure it has sufficient automatic tests. There is no test department doing that.

No testing phase = no testers?

If the testing phase doesn’t exist anymore, where are the testers? And that’s (one of the) reason(s) why I think a lot of testers don’t like Agile development.

I believe that testers will need a different skill set than now. They will have to work more closely with customers - helping them discover untested areas. Helping them break down big ideas into smaller ideas. Helping programmers figure out a way to test something automatically, that is particularly hard to test automatically.

But once they do all that - they aren’t testers anymore. They are team members. And that’s what they always should have been.

Funny definition of “iteration”

, No Comments

From one of the meetings I attended:”Iteration end is 7th of September or one week later - or two weeks later. Or three weeks later, depending on how many more weeks we need.”

Observations on why we get paid (’Why we get paid’-Series)

October 20, 2007, No Comments

Here are some observations on why we get paid.

From a Development Company’s perspective:

• Making A smaller increases the number of projects that make economical sense
• Making B bigger means giving the customer a competitive advantage
• Making B bigger is what Software Development companies generally don’t care about
• New technology has to either make A smaller or B bigger or both.

From a Customers perspecive:

• Trying to make A smaller is what outsourcing is about
• B is harder to quantify than A. And A is already very hard to quantify.
• Customers don’t spend enough time quantifying B.
• Optimizing A doesn’t make sense if you sacrifice B

Do you see any others? I’d love to hear them in the comments!

Big projects are just a bunch of little pieces

October 13, 2007, No Comments

Breaking down requirements

One of the most underrated skills in developing software is to break requirements down into smaller pieces. You can do that several ways. For example you can conclude that to be able to write the program, you need to have the database in place. So you start with database design. Then you go on to write the business logic on top of it. Finally you add the user interface. Let me call these pieces technical slices.Another way would be to slice by functionality. For example we start with the administration tool, so data can be added to the system. Then we write the functionality for the shop. Then for shipping the items. etc.Very often those slices appear to have dependencies on others. For example:

• Can you write the business logic without the database?
• To be able to sell items, there has to be a way to add items to the system

For agile development it’s crucial to be able to break down ideas into very small pieces of independent functionality. Why? Because you want to let the customer chose what is most valuable to him and work on that. If the pieces of functionality (called “user stories” in Extreme Programming), aren’t independent the customer has no choice.Little pieces of functionality means it can be implemented and ready to use in about 2 weeks or less. Finding those pieces is a new skill - it isn’t taught at schools.Breaking the requirements down into little pieces has a lot of advantages but also some disadvantages.

Advantages

Little pieces of functionality…

• …are done quickly, giving the team a lot of small successes - increasing motivation

• …minimize risk, because you can do the tough stuff first and find problems early

• …help the project make money fast, because you can have a valuable subset of functionality ready for use early, decreasing your time to market

• …enable changes in requirements, because things that aren’t implemented don’t have to be and new ideas can make it to production fast.

• …increase visibility of the projects progress (e.g. Running Tested Features)

• …help you recognize problems in performance, usability and even help you discover misunderstandings. Early, of course.

• …are easier to test (because there is less to test) and easier to test automatically (because they are independent)

Disadvantages

Little pieces of functionality…

• …never stop coming in, making it hard to put a price tag on a project at the very beginning

• …never stop coming in, making it hard to say when you’re done with everything

• …change with every thing the customer learns during the project, making it impossible to tell the customer what he’s going to get in the end

• …work bad with a rewrite, because well, it just has to do what the old software already did.

A lot of the disadvantages are strongly dependent on the structure of the contract. A lot of customers (and IT people) look at building software the same way as building a house. They want to know how the house looks, they want to know when it’s done and how much it costs. This doesn’t work very well with houses - and it doesn’t work at all with software.

The thorny road to Working Software

September 29, 2007, No Comments

What do you have to do to be sure your code works?

That’s the best case. Write Code. Test Code. The less time it takes to test the code, the more often I actually test the code. If I could test all my code in one second, I’d test after every line of code I write. Or even better: the IDE would run the tests after every line of code and inform me if something unexpected happens.

That was the ideal case. Sometimes that cycle looks more like this:

It takes more time to test the code. Therefore I write more code before I test it - increasing the likelyhood of doing something wrong and having to repeat the cycle. Using a good debugger may help shorten that cycle.

Interestingly the debugger isn’t needed in the ideal scenario, because it doesn’t shorten the cycle there. In some environments it’s almost impossible to live without a debugger - but you should always question the use of it. Using the debugger is always a sign that I’m not working in the ideal cycle. And instead of keep on working, thoughts about how I develop software are necessary. Dependence on the debugger may be the reason why it’s hard to create working software - even if it seems to make it easier at first glance.
Bigger projects usually add more complexity. So the cycle looks more like that:

Depending on your application and environment, Hot Code Replacement can take you to one of many places. Ideally the code is replaced and is called the next time you use your application. Sometimes you have to restart your application, because the code you changed only gets called on startup, or it needs a clean state. Other times you need to restart the server, because your changes only get executed at server startup.

Of course, sometimes Hot Code Replacement fails. If you’re lucky, your IDE will tell you. Sometimes your IDE gets confused (seems to be the permanent state of RAD) and won’t replace your code without warning. That sucks because your code changes didn’t do anything. If it happens too often, you get annoyed and decide to always restart the server - just to be sure.

Here is an example from a project I worked on:

If you have to do a full circle, it takes about 3 minutes. The best case is 20 seconds, which is hardly ever the case because of the GUI framework choice.

These long waits lead to something else

Multitasking.

This is one reason why it takes so long to create enterprise applications. It’s also a reason why it’s hard to create working software in enterprises. Very often, enterprise applications just aren’t built to be tested.

It’s also interesting to note, that so far the complexity of the application to be developed hasn’t even been mentioned.

That’s why I don’t like Application Servers.

10 Reasons why creating a program takes longer than it should

June 27, 2007, No Comments

The following is a list of annoying timewasters that I seem to stumble on over and over again… Bigger projects face more of these potential timewasters, making estimates more inaccurate.

10 - Coding Style Issues: I’ve had discussions about where to place {, whether it is better to have long lines on 1 line (and use the scrollbar) or on several lines. About prefixes for variables. The list goes on and on. The fun part about the discussion is: Nobody listens to your arguments, because it’s all about personal preference. Why can’t we just all agree on my style?

Tools like Checkstyle seem to magnify the problem rather than make it disappear.

9 - MS Word - An extremly poor tool to communicate requirements.

8 - Deploying: How much time should be between successfully testing a feature on the developers machine and getting that feature on a server for everyone to try out? In some places, the real work just begins at that point…

7 - Mapping Data, Data migration: Whenever you get in touch with data that needs to be moved from one system to another, you’re basically doomed. Services (of the SOA kind) can be just as painful. There is so much “data mapping” going on in Enterprise environments that I think it would make sense to find a new Job Title for the ‘Datamapper’.

6 - Strange Server Behaviour: Things that should just work, don’t. Of course, sometimes they do. The problem usually lies in your code - well, that doesn’t make it easier, because you think the problem lies somewhere else. For example: This is wrong. So wrong.

5 - classpath/dll mess: I don’t remember how long it took to find out exactly which dll made Crystal Reports crash. But it wasn’t fun to find out and it was more than a day. The same applies to “commonly used java jar files”, which are copied all over your system - often in different versions. It’s almost guaranteed you have the wrong one in your classpath.

4 - Useless Errormessages: Try integrating MS Office. No matter what you do, if something goes wrong a very helpful message appears. Like ‘Error’. Or my alltime favorite from the Jet Engine: “Reserved Error - There is no message for this error.”. Nothing beats a little humor in your daily life - especially when the deadline is close…

3 - Defects (aka bugs): Creating a feature is sometimes rather trivial. So is it justified to spend 1 h to write unit tests for a feature you can implement in 5 minutes? Well, you can always skip it and then spend 3 h testing and 3 h debugging next week, when sombody else broke your feature. Good thing the fix only takes 5 minutes (never mind that an other feature might be broke by the “fix”).

2 - Communicating with other systems/programs/services: It all looks nice and harmless. Always. OK - You have to map data, which can be a problem (see point 7).

If something doesn’t go right (and it never does) you sit there with no possibility to find out why - because you don’t have access to the logs of the other system. If you’re lucky you get an error (See point 4). Sometimes you’re not lucky and you just sit there and the stupid thing doesn’t do anything.

And don’t even get me started on Character Set Encodings. Retrieving data from an other system: 1 h. Figuring out why you get a ‘?’ instead of an ‘€’ sign: rest of the week.

1 - Bad Frameworks: First you spend time learning what the framework can do. Then you start doing a project and learn what it can’t do. Then you spend several hours trying to do something that should be trivial, just because the framework prevents you from doing it the trivial way.

Honorable mention goes to:

11 - Missing methods/Overdesigned classes: Give me one good reason why this


cal = new GregorianCalendar(2006, 1, 31);


creates an instance of the calendar set to the 3rd of March 2006.

And as a bonus

12 - Getting the GUI exactly right. Most of the things look and behave just like you’d like. But the few things that don’t, take forever until they do.

[UPDATE]:

13 - Reports: You can spend so much time on creating nice reports. To make things even more fun: The reports are usually some of the most important things to come out of the system, because they will often be sent to the customers. Therefore they have to be perfect. Oh. And usually work on the reports starts after everything is finished. So time is limited.

Testing unclear requirements

June 10, 2007, No Comments

In Lukas’ talk there he pointed out a bad requirement, that would be impossible to test because it lacks details:

The instrument shall provide the capability for authorized users to create, edit and load files.

What do you do about a requirement that is vague?

One way is to raise your hand and tell people that this requirement isn’t testable. It needs more detail. Have them write it down in a document.

Another one is to just write test cases for it. Like this:


Try to login as Administrator and create a tst file.

| enter | username | admin |
| enter | password | geheim |
| press | login |
| check | isInRole | Administrator |
| check | countFiles | 0 |
| enter | createFile | newTestFile.tst |
| check | countFiles | 1 |
This is an almost-valid fitnesse test, that tries to login, makes sure the person is in the administrator role and tries to create a file. Similar ones are needed for somebody who doesn’t have access rights. For editing and loading.

The requirement quickly becomes less important, because the details are in the tests. The tests have to be in an easily readable format. By readable I mean:

The customer (a nontechnical person) has to be able to look at it and say whether this is what he wants or not. If he doesn’t like it, he should be able to change it or enhance it.

To further limit possible test tools, add to the above:

It has to be possible to specify the tests before any software exists.

And:

It should be little or no effort for the programmers to connect production code to the test specification.

Is there a tool that allows for all that? I don’t know. I guess fitnesse comes closest. But there is a lot of room there…

As you can see, the requirements can be vague. But with the details documented in automated test cases you are in a much better shape than with a BLOB called requirements document.

Another question is: Do you really need to write down those details at the beginning of the project?