Home > Spock Framework > What’s New In Spock 0.4? Episode 1: Data Tables

What’s New In Spock 0.4? Episode 1: Data Tables

With the Spock Framework 0.4 release around the corner, I thought it was time to demonstrate some of the upcoming new features (which are already available in recent snapshots). So here comes the first episode in the mini-series “What’s New In Spock 0.4”. Today’s topic is data tables, a new way to write data-driven test methods.

Data-driven testing has always been one of Spock’s strong points. To give you some background, let’s have a look at a typical example taken from the spock-example project:

def "maximum of two numbers"() {
  expect:
  Math.max(a, b) == c

  where:
  a << [3, 5, 9]
  b << [7, 4, 9]
  c << [7, 5, 9]
}

This method is testing the Math.max() operation. The expect block contains the test logic (what we expect from the Math.max() operation), and the where block contains the test data. The key to interpreting the where block is to read it from top to bottom. That is, variables a, b, and c will be assigned values 3, 7, and 7 for the first run of the method; values 5, 4, and 4 for the second run; and values 9, 9, and 9 for the third run. We say that the method has three iterations.

Data-driven test methods are a powerful tool, but the where block syntax shown above suffers from two problems:
1. It is a bit noisy (shift operators, brackets, commas)
2. It is read from top to bottom, which is less intuitive than reading from left to right (at least for people used to left-to-right languages).

This is where data tables come in. Their purpose is to provide a convenient syntax for in-line test data. Let’s modify the previous example to use a data table:

def "maximum of two numbers"() {
  expect:
  Math.max(a, b) == c

  where:
  a | b | c
  3 | 7 | 7
  5 | 4 | 5
  9 | 9 | 9
}

Run this example in Spock Web Console

Now the where block reads like a table. The first row (the table header) specifies the data variables, and the remaining rows specify their values for each iteration. Compared to the previous example, this one is both nicer to read and write.

As demonstrated in their specification, data tables cannot just hold literals but also more complex expressions. However, other forms of parameterizing a test method (like the one we saw in the first example) haven’t lost their value. For example, loading data from external sources is still best left to multi-parameterizations:

where:
[a, b, c] << sql.rows("select a, b, c from maxdata")

See DatabaseDriven for the full example.

Apart from the different syntax, data tables behave like all other forms of parameterizations. For example, their data variables can be optionally declared as method parameters, and they can be combined with derived parameterizations. To learn more about the different forms of parameterizations, see their documentation.

That’s it for the first episode of “What’s New In Spock 0.4”. Please tune in tomorrow for the second episode. Until then, I’ll leave you with a real-world usage of data tables taken straight from the Sales Intelligence Engine codebase:

@Unroll
def "determine dominant color"() {
  when:
  action.image = image
  action.ignoreWhite = ignoreWhite	
  action.execute()
	
  then:
  action.dominantColor == dominantColor		
			
  where:
  image                   | ignoreWhite | dominantColor 
  '/images/white.png'     | false       | 'ffffff'
  '/images/black.png'     | true        | '000000'
  '/images/28843_300.jpg' | true        | 'ffcc33' 
  '/images/20341_300.jpg' | true        | 'cc6666'
  '/images/20692_300.jpg' | true        | 'cccccc' 
  '/images/7870_300.jpg'  | true        | '993333'
}
Categories: Spock Framework
  1. March 11, 2010 at 13:22

    What is the benefit over

    expect:
    max(6, 7) == 7
    max(7, 7) == 7
    max(7, 6) == 7

    ?

    • March 11, 2010 at 13:48

      @Dierk: The main benefit of data-driven test methods is that test data is clearly separated from test logic. In the Math.max() example it doesn’t make a big difference, but in the last example (“determine dominant color”) it’s already more apparent.

  2. March 11, 2010 at 16:31

    Well, classicly, in the last example, the when:/then: parts would go into an assertion method and the test would read like

    assertDominantColor ‘a.jpg’, true, ‘ffffff’
    assertDominantColor ‘b.jpg’, false, ‘ffffff’

    where the data is equally well separated, I would say.
    Plus I have the benefit of “no-magic” and when an assertion fails, the stacktrace points me to the right place (second from top)…

    Or do I miss anything obvious?

    • March 11, 2010 at 17:01

      >Well, classicly, in the last example, the when:/then: parts would go into an assertion method
      when-then has a lot more semantics than a “classic assertion method”. It’s the basis for mocking, testing for exceptions, and other Spock features. However, when-then can’t be moved into a helper method as a whole (only a block’s content can).
      >Plus I have the benefit of “no-magic”
      It’s a Spock language feature, so I don’t consider it magic (although it takes a bit of effort to learn).
      > and when an assertion fails, the stacktrace points me to the right place
      Data-driven test methods can do even better. With @Unroll, every iteration is visible in JUnit reports and the IDE JUnit runner (passed/failed). In the future you might be able to re-run just a particular iteration.

      There are other reason why data-driven methods can be a better solution, for example:
      – Data-driven methods let you easily generate test data. Example: “[x,y] << MyEnum1.values().combinations(MyEnum2.values())"
      – The fact that the where-block comes last lets you first focus on test logic, without worrying about test data or how to structure your test (assertion method)
      – Iterations are treated as independent tests (setup()/cleanup() is called for each of them)
      – Data-driven methods can easily load external test data, e.g. 10.000 data sets from a CSV file. Here the separation I talked about becomes critical.
      – Spock runtime and extensions are aware of iterations and thus can provide additional insights/services

      That said, in simple cases your proposal is just fine too.

    • March 11, 2010 at 17:17

      By the way, the related patterns from the xUnit Patterns book are:
      – Parameterized Test http://xunitpatterns.com/Parameterized%20Test.html
      – Data-Driven Test http://xunitpatterns.com/Data-Driven%20Test.html

  3. March 11, 2010 at 17:31

    Ok, thanks a lot for the clarification!

  4. March 11, 2010 at 23:24

    Hi, out of curiosity, did you came up with this idea and syntax on your own or having a look at specs: http://code.google.com/p/specs/wiki/AdvancedSpecifications?

    In any case I’m happy that this syntax is pushed forward in the BDD community/tools as I think that it makes sense when you have lots of small and similar examples.

    Eric.

  5. March 23, 2010 at 05:49

    > By the way, a few months ago I ported your edit distance code

    Nice implementation! I was also thinking of changing my code based on that post I’ve read recently:

    http://oldfashionedsoftware.com/2009/11/19/string-distance-and-refactoring-in-scala

    Eric.

  6. Nicholas White
    March 27, 2010 at 01:34

    Do you have any idea when 0.4 will be released? I think data tables really are (will be 🙂 spock’s ‘killer feature’…

    Thanks,
    Nick

    • April 2, 2010 at 00:32

      Hi Nick,

      Spock 0.4 will be released next week. (No, this isn’t an April Fools’ joke.)

      Cheers,
      Peter

  7. Marty
    April 13, 2010 at 11:45

    Hi Peter,

    is there any reason you are using the syntax

    when – then – where

    from a user point of view I would like to read

    given – when – then

    so the actual data at the top (given), then the action (when) and the assumption (then)…

    This is actually the only thing which bothers me with Spock 😉

    Thanks
    Marty

    • April 13, 2010 at 22:15

      As you may know, Spock does support the given-when-then syntax, with “given” being an alias for “setup”. However we felt that for data-driven tests a separate block was needed, which we called “where” (inspired from Haskell).
      The where-block is very different from all other blocks. It is only evaluated once per data-driven method, whereas all other blocks are evaluated once per iteration. It can only access @Shared fields, whereas all other blocks can access both instance and @Shared fields. At the same time, it is closely related to the other blocks in that it provides them with data. Therefore we wanted to have a way to define it along with them rather than putting it into a separate method (like it’s done in JUnit and TestNG). The where-block comes last because we felt that this would set it better apart and result in a nice flow (first define the test’s logic and only then it’s data).
      Yet another argument for a separate block is that a where-block can parameterize every part of a test method (often it will also describe expected outcomes), whereas the purpose of a given/setup block is to establish the context (fixture) for a test. So even when following a pure TDD/BDD approach I think that parameterizations shouldn’t go into the given-block.

  8. kirk
    May 13, 2010 at 00:40

    I love the spock plugin but it is not compatible with Grails-1.3.0/Groovy-1.7.2.
    Are there plans to upgrade the plugin?

    • May 13, 2010 at 01:05

      There is already a version for Grails 1.3. Have you tried “grails install-plugin spock 0.4-groovy-1.7-SNAPSHOT”?

      • kirk
        May 13, 2010 at 18:19

        No, will try, thanks!

      • kirk
        May 13, 2010 at 19:44

        Works good.
        Any idea if there will be IDEA integration any time soon?
        The plugin currently does not compile using IDEA and the spock tests can’t be run by the test-runner.
        Thanks.

      • May 13, 2010 at 19:54

        I suppose this is a problem with IDEA’s Grails integration. Regular Spock tests work just fine in IDEA. But I’ll have a look.

        PS: Please use http://forum.spockframework.org for future discussions.

      • June 13, 2010 at 15:27

        @Kirk: I’ve filed an IDEA issue for this (http://youtrack.jetbrains.net/issue/IDEA-55330), and it’s already fixed in the latest EAP’s.

  1. March 12, 2010 at 05:05
  2. March 14, 2010 at 12:14
  3. May 4, 2010 at 08:44

Leave a reply to pniederw Cancel reply