What’s New In Spock 0.4? Episode 1: Data Tables
With the Spock Framework 0.4 release around the corner, I thought it was time to demonstrate some of the upcoming new features (which are already available in recent snapshots). So here comes the first episode in the mini-series “What’s New In Spock 0.4”. Today’s topic is data tables, a new way to write data-driven test methods.
Data-driven testing has always been one of Spock’s strong points. To give you some background, let’s have a look at a typical example taken from the spock-example project:
def "maximum of two numbers"() { expect: Math.max(a, b) == c where: a << [3, 5, 9] b << [7, 4, 9] c << [7, 5, 9] }
This method is testing the Math.max() operation. The expect block contains the test logic (what we expect from the Math.max() operation), and the where block contains the test data. The key to interpreting the where block is to read it from top to bottom. That is, variables a, b, and c will be assigned values 3, 7, and 7 for the first run of the method; values 5, 4, and 4 for the second run; and values 9, 9, and 9 for the third run. We say that the method has three iterations.
Data-driven test methods are a powerful tool, but the where block syntax shown above suffers from two problems:
1. It is a bit noisy (shift operators, brackets, commas)
2. It is read from top to bottom, which is less intuitive than reading from left to right (at least for people used to left-to-right languages).
This is where data tables come in. Their purpose is to provide a convenient syntax for in-line test data. Let’s modify the previous example to use a data table:
def "maximum of two numbers"() { expect: Math.max(a, b) == c where: a | b | c 3 | 7 | 7 5 | 4 | 5 9 | 9 | 9 }
Run this example in Spock Web Console
Now the where block reads like a table. The first row (the table header) specifies the data variables, and the remaining rows specify their values for each iteration. Compared to the previous example, this one is both nicer to read and write.
As demonstrated in their specification, data tables cannot just hold literals but also more complex expressions. However, other forms of parameterizing a test method (like the one we saw in the first example) haven’t lost their value. For example, loading data from external sources is still best left to multi-parameterizations:
where: [a, b, c] << sql.rows("select a, b, c from maxdata")
See DatabaseDriven for the full example.
Apart from the different syntax, data tables behave like all other forms of parameterizations. For example, their data variables can be optionally declared as method parameters, and they can be combined with derived parameterizations. To learn more about the different forms of parameterizations, see their documentation.
That’s it for the first episode of “What’s New In Spock 0.4”. Please tune in tomorrow for the second episode. Until then, I’ll leave you with a real-world usage of data tables taken straight from the Sales Intelligence Engine codebase:
@Unroll def "determine dominant color"() { when: action.image = image action.ignoreWhite = ignoreWhite action.execute() then: action.dominantColor == dominantColor where: image | ignoreWhite | dominantColor '/images/white.png' | false | 'ffffff' '/images/black.png' | true | '000000' '/images/28843_300.jpg' | true | 'ffcc33' '/images/20341_300.jpg' | true | 'cc6666' '/images/20692_300.jpg' | true | 'cccccc' '/images/7870_300.jpg' | true | '993333' }
What is the benefit over
expect:
max(6, 7) == 7
max(7, 7) == 7
max(7, 6) == 7
?
@Dierk: The main benefit of data-driven test methods is that test data is clearly separated from test logic. In the Math.max() example it doesn’t make a big difference, but in the last example (“determine dominant color”) it’s already more apparent.
Well, classicly, in the last example, the when:/then: parts would go into an assertion method and the test would read like
assertDominantColor ‘a.jpg’, true, ‘ffffff’
assertDominantColor ‘b.jpg’, false, ‘ffffff’
…
where the data is equally well separated, I would say.
Plus I have the benefit of “no-magic” and when an assertion fails, the stacktrace points me to the right place (second from top)…
Or do I miss anything obvious?
>Well, classicly, in the last example, the when:/then: parts would go into an assertion method
when-then has a lot more semantics than a “classic assertion method”. It’s the basis for mocking, testing for exceptions, and other Spock features. However, when-then can’t be moved into a helper method as a whole (only a block’s content can).
>Plus I have the benefit of “no-magic”
It’s a Spock language feature, so I don’t consider it magic (although it takes a bit of effort to learn).
> and when an assertion fails, the stacktrace points me to the right place
Data-driven test methods can do even better. With @Unroll, every iteration is visible in JUnit reports and the IDE JUnit runner (passed/failed). In the future you might be able to re-run just a particular iteration.
There are other reason why data-driven methods can be a better solution, for example:
– Data-driven methods let you easily generate test data. Example: “[x,y] << MyEnum1.values().combinations(MyEnum2.values())"
– The fact that the where-block comes last lets you first focus on test logic, without worrying about test data or how to structure your test (assertion method)
– Iterations are treated as independent tests (setup()/cleanup() is called for each of them)
– Data-driven methods can easily load external test data, e.g. 10.000 data sets from a CSV file. Here the separation I talked about becomes critical.
– Spock runtime and extensions are aware of iterations and thus can provide additional insights/services
That said, in simple cases your proposal is just fine too.
By the way, the related patterns from the xUnit Patterns book are:
– Parameterized Test http://xunitpatterns.com/Parameterized%20Test.html
– Data-Driven Test http://xunitpatterns.com/Data-Driven%20Test.html
Ok, thanks a lot for the clarification!
Hi, out of curiosity, did you came up with this idea and syntax on your own or having a look at specs: http://code.google.com/p/specs/wiki/AdvancedSpecifications?
In any case I’m happy that this syntax is pushed forward in the BDD community/tools as I think that it makes sense when you have lots of small and similar examples.
Eric.
Hi Eric,
probably a bit of both. In my recent experiments, this syntax turned out as the winner. On the other hand, I always keep an eye on RSpec, Specs, and other frameworks, and I’ve seen Specs’ table syntax before.
By the way, a few months ago I ported your edit distance code to Spock/Java, then rewrote some parts of it. See http://code.google.com/p/spock/source/browse/#svn/trunk/spock-core/src/main/java/org/spockframework/runtime/condition and http://meet.spockframework.org/?id=4001
A final note: I don’t see Spock as a BDD tool, but more as a general-purpose testing/spec framework (say like ScalaTest).
Cheers,
Peter
> By the way, a few months ago I ported your edit distance code
Nice implementation! I was also thinking of changing my code based on that post I’ve read recently:
http://oldfashionedsoftware.com/2009/11/19/string-distance-and-refactoring-in-scala
Eric.
Do you have any idea when 0.4 will be released? I think data tables really are (will be 🙂 spock’s ‘killer feature’…
Thanks,
Nick
Hi Nick,
Spock 0.4 will be released next week. (No, this isn’t an April Fools’ joke.)
Cheers,
Peter
Hi Peter,
is there any reason you are using the syntax
when – then – where
from a user point of view I would like to read
given – when – then
so the actual data at the top (given), then the action (when) and the assumption (then)…
This is actually the only thing which bothers me with Spock 😉
Thanks
Marty
As you may know, Spock does support the given-when-then syntax, with “given” being an alias for “setup”. However we felt that for data-driven tests a separate block was needed, which we called “where” (inspired from Haskell).
The where-block is very different from all other blocks. It is only evaluated once per data-driven method, whereas all other blocks are evaluated once per iteration. It can only access @Shared fields, whereas all other blocks can access both instance and @Shared fields. At the same time, it is closely related to the other blocks in that it provides them with data. Therefore we wanted to have a way to define it along with them rather than putting it into a separate method (like it’s done in JUnit and TestNG). The where-block comes last because we felt that this would set it better apart and result in a nice flow (first define the test’s logic and only then it’s data).
Yet another argument for a separate block is that a where-block can parameterize every part of a test method (often it will also describe expected outcomes), whereas the purpose of a given/setup block is to establish the context (fixture) for a test. So even when following a pure TDD/BDD approach I think that parameterizations shouldn’t go into the given-block.
I love the spock plugin but it is not compatible with Grails-1.3.0/Groovy-1.7.2.
Are there plans to upgrade the plugin?
There is already a version for Grails 1.3. Have you tried “grails install-plugin spock 0.4-groovy-1.7-SNAPSHOT”?
No, will try, thanks!
Works good.
Any idea if there will be IDEA integration any time soon?
The plugin currently does not compile using IDEA and the spock tests can’t be run by the test-runner.
Thanks.
I suppose this is a problem with IDEA’s Grails integration. Regular Spock tests work just fine in IDEA. But I’ll have a look.
PS: Please use http://forum.spockframework.org for future discussions.
@Kirk: I’ve filed an IDEA issue for this (http://youtrack.jetbrains.net/issue/IDEA-55330), and it’s already fixed in the latest EAP’s.