Better IO testing through Monads

Unit Testing is a great way to try to verify consistent functionality over the life of our code. Typically we design our tests to cover ranges and edge cases of our code making sure that we can handle all inputs. This is made easy when we write in Functional Languages as there (should be) no side effects. But when you **want** to deal with IO and side affect code in Haskell, this becomes much more difficult.

This problem has been fixed many times, but often with very complex solutions. Let's see if we can make it really simple (even if the post is long).

If you want to play along the code is availabe from git or [zip file][7]

git clone https://commentedcode.org/git/blog/mockio.git

The Problem

While learning how to use [fltkhs][1], and creating a GUI [Diceware][2] password generator, I realized that I was going to be doing a lot of work in the `IO` Monad. For the GUI parts I didn't really care to do a lot of unit testing so I didn't think there would be much of an issue.

When we think of `IO` we often think of interacting with the User.

helloWorld :: IO ()
helloWorld = do
  putStr "What is your name? "
  name <- getLine
  putStrLn $ "Hello " ++ name

The `IO` Monad is used for other interactions such as random number generation. Here is a simple function that simulates rolling an "n-sided" die.

rollDice :: Int -> IO Int
rollDice sides = getStdRandom $ randomR (1,sides)

Using the repl we can see that our function returns numbers, but since its random we can't really see if the range of outputs are valid or not.

>>> rollDice 6
5
>>> rollDice 6
3
>>>

What makes this even worse is that because the value returned is random, doing any sort of edge case testing on code that uses this function is nearly impossible.

Mocks make simulation easier

Per [Wikipedia][2]

In object-oriented programming, mock objects are simulated objects that mimic

the behavior of real objects in controlled ways. A programmer typically

creates a mock object to test the behavior of some other object

Even though we are not working with a OO language, we can still take the concept of a Mock Object and apply it here. What better use for Monads than to create simulated scenarios to mock possible real world use cases.

So why Monads?

The feature of Monads we will be using deals with the fact that they allow us to temporarily modify the normal rules of haskell. This, of course, is done in a controlled way as to keep things still within the realm of typical Haskell and FP.

Being a pure functional language, Haskell functions are [referentially transparent][3]. This means that no matter the state of the application or the system it runs on, a function will always return the same result given an input. `sin()` will always return `0` when given the value `pi`. `toUpper` always returns `"HELLO"` when given `"hello"`.

Think about what rules change in haskell when you are in the `IO` Monad. When you call `getLine` the string returned is dependent on the user at the keyboard and two individuals may type two different values. This means that `getLine` is not referentially transparent. Within the context of the `IO` monad, its OK that this function doesn't follow Haskell's rules.

Haskell is also a (mostly) immutable language. New values are stored under new names, while old ones are cleaned up. Counters in loops don't exist, but instead are passed in as a counting argument into a recursive function. Updating values just isn't done.

But when you introduce the `State` Monad the rule of immutability is no more. Again, its a controlled manipulation of the rules, but you are able to change the application's state through a mutable object.

So what rules do we want to bend for our testing purposes?

The Testing Problem, Type Classes & Mocking

To repeat our problem, when running a unit test we can't rely on non-referentially transparent functions like `getLine` and `randomR` to give us consistent values. Because of this, it would be possible for a test to fail on one run, and be successful on another. This does not make for good testing.

What if we could write our code that uses the `IO` monad when we were running normally, but then run a special monad that mock's the `IO` monad when we do our testing? Then we could produce consistent results with our unit tests without having to redesign our code, or do modifications between runtime and test time.

Type Classes

This is where [Type Classes][4] come in. Type Classes are a way for Haskell to provide a polymorphic interface to types. You can define a set of functions and group them into a type class. Then you can declare instances of the type class with data types that exist, providing the implementation of its group of functions with relation to the instance that data type.

The `Eq` type class is used for testing equality. Members of the type class must provide `(==)` and `(/=)` implementations so that you can compare two value to be equal or not.

-- Person: Name Age
data Person = Person String Int

instance Eq Person where
  -- Both Persons are equal if they have the same Name and Age
  (Person n1 a1) == (Person n2 a2) = n1 == n2 && a1 == a2

The `Ord` type class is used for ordering items, with the following definition:

class  (Eq a) => Ord a  where
  compare              :: a -> a -> Ordering
  (<), (<=), (>=), (>) :: a -> a -> Bool
  max, min             :: a -> a -> a

Here we see that all members of the `Ord` type class must provide implementation to a hand full of different functions used for ordering. On the first line we also see `(Eq a)` which tells us that all types that are an instance of `Ord` must also be an instance of `Eq`. This makes sense since you'd be able to use equality during your ordering process.

Even monads are represented as type classes.

class Applicative m => Monad (m :: * -> *) where
  (>>=) :: m a -> (a -> m b) -> m b
  (>>) :: m a -> m b -> m b
  return :: a -> m a
  fail :: String -> m a

All Monads are Applicatives, with bind, return, and fail functions.

Making our own Type Class

Let's create our own type class. One that requires all the IO functions we will be using in our code that might affect our unit testing.

For our first operation we'll write a string to an output. In the `Prelude` library we have the following function:

putStr :: String -> IO ()

A `String` is passed to the function, the value is written to stdout and then returns unit `()`...all occurring within the context of the `IO` monad. What we need to do is define our type class that works both within the `IO` monad as well as other monads. Since `Monad` is a type class, we can write our type class to only allow derived types to be part of the `Monad` class (just like `Ord` and `Eq`).

class Monad m => MockIO m where
  mPutStr :: String -> m ()

Our function `mPutStr` takes a `String` as an argument and then does "something" within the context of the `m` monad and returns unit `()`. In the `IO` monad we want that "something" to be writing to stdout. And since the signature of our function matches that of `putStr` we can easily define that for our `IO` instance of `MockIO`:

instance MockIO IO where
  mPutStr = Prelude.putStr

We can do the same for `putStrLn` and `getLine`. For our random number generator we'll define it to make our dice rolling simpler.

-- src/MockIO.hs
module MockIO where

class Monad m => MockIO m where
  mPutStr :: String -> m ()
  mPutStrLn :: String -> m ()
  mGetLine :: m String
  mGetRandomInt :: Int -> Int -> m Int
  mPutStrLn s = mPutStr $ s ++ "\n"

Here is the runtime implementation for `IO`.

-- src/MockIO/IO.hs
module MockIO.IO where

import System.Random
import MockIO

-- |Instance of MockIO for runtime execution with IO
instance MockIO IO where
  mPutStr = Prelude.putStr
  mPutStrLn = Prelude.putStrLn
  mGetLine = Prelude.getLine
  mGetRandomInt min max = getStdRandom (randomR (min, max))

So how do we use this new MockIO?

Let's go back to our dice rolling code and rewrite it using our new `MockIO` type class.

rollDice :: MockIO m => Int -> m Int
rollDice sides = mGetRandomInt 1 sides

Pretty simple. We define our type `m` as an instance of `MockIO` and then use functions from within that type class. If we load this code up in the repl we get:

>>> import MockIO
>>> Import MockIO.IO
>>> Import Dice
>>> rollDice 6
4
>>> rollDice 6
6
>>> rollDice 6
2
>>>

Since the repl is always running within the `IO` monad, we use the `IO` instance functions.

Mutable State as IO

So where does this get us? We now have the ability to call `IO` functions when we are working within `IO`. But in unit testing we want to remove the random results of `IO` with something fixed.

The `State` monad allows us to create a mutable state variable that exists during the context of the monad. We can use this monad to store expected values and return them as if they were pulled from IO. A simple explanation of `State` can be found at [LYaH][5].

The first step is to create the data type that will store our state. We want to provide fixed values for stdin and for random numbers, and it would be nice to store off things written to stdout.

data MockState = MockState
  { outputs :: [String] -- Strings the mock will output, like getLine
  , inputs  :: [String] -- Strings the mock will input, like putStr
  , outputInts :: [Int] -- "Random" ints we will return in mGetRandomInt
  }
  deriving (Show, Eq)

Pretty simple, just a set of arrays contained inside a data type. We will derive `Show` and `Eq` type classes as to make debugging easier.

`State` has a type definition of

newtype State s a = State { runState :: s -> (a,s) }

where `s` is the state information and `a` is the return type. The `runState` function is used to execute a function within the context of the `State` monad. Our use case with our new state object will be:

State MockState a = State { runState :: MockState -> (a,MockState) }

Rather than always typing `State MockState` we can define our own type:

type SimpleMockedIO = State MockState

runMockIO :: SimpleMockedIO a -> MockState -> (a, MockState)
runMockIO = runState

For our unit test setup we will use a `MockState` to setup our environment and then execute the code using these new functions and data types. For example, we can use our dice rolling function and test that it returns the correct value.

We call `rollDice` within the `SimpleMockedIO` context giving the following function definition and we see the change:

>>> :t rollDice
rollDice :: MockIO m => Int -> m Int
>>> :t runMockIO (rollDice 6)
runMockIO (rollDice 6) :: MockState -> (Int, MockState)
>>>

Given a `MockState` this curried function would return an `Int` and the mutated state.

>>> let env = MockState [] [] [1] -- Ouputs, Inputs and Random Numbers
>>> let (result, _) = runMockIO (rollDice 6) env
>>> result
1
>>>

We defined the value returned from our "random number generator" and sure enough that is the value. Lets test it even more.

>>> let env = MockState [] [] [1,2,3,4,5,6]
>>> :{
let test :: MockIO m => m [Int]
    test = do
      r1 <- rollDice 6
      r2 <- rollDice 6
      r3 <- rollDice 6
      r4 <- rollDice 6
      r5 <- rollDice 6
      r6 <- rollDice 6
      return [r1, r2, r3, r4, r5, r6]
:}
>>> let (res,_) = runMockIO test env
>>> res
[1,2,3,4,5,6]
>>>

Six rolls and six exactly as expected results. So how is this done?

Pretend to be IO

Just like we did with the `IO` monad making it an instance of `MockIO`, we need to do the same with our new state monad `SimpleMockedIO`. For each of the functions we need to retrieve the current state, do whatever IO is necessary and then update the state for future use. Let's take a look at how the random numbers are handled.

instance MockIO SimpleMockedIO where
  mGetRandomInt min max = do
    st <- get
    case outputInts st of
      [] -> error "MOCK ERROR: Empty random number generator"
      (i:ix) -> do
        if i < min || i > max
          then error "MOCK ERROR: Bad Values in Random Number Generator"
          else do
            let newSt = st { outputInts = ix }
            put newSt
            return i

We set the current state to `st` and then check to see if we have any numbers left in our mock environment. We check to make sure the next value is within our min and max (as we assume the way `IO` handles this would always work). Lastly we update the state by removing the number from the list, saving the new state off and returning our "random" number.

Writes to stdout are handled by concatenating to a list and reads are done in a similar way as the random numbers, just with `String` instead of `Int`.

-- src/MockIO/Simple.hs
{-# LANGUAGE TypeSynonymInstances #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE InstanceSigs #-}

module MockIO.Simple where

import Control.Monad.State
import Data.Functor.Identity
import MockIO

-- | MockState is a data structure used for simple mocking of IO
-- This data type contains intput, and output strings to simulate stdin and stdout
-- as well as a list of integers for a random number generator.
data MockState = MockState
  { outputs :: [String] -- getLine
  , inputs :: [String]  -- putStr(Ln)
  , outputInts :: [Int] -- mGetRandomInt
  }
  deriving (Show, Eq)

-- |SimpleMockedIO is the data type that simulates the runtime environment
-- See runMockIO for details on simulating IO
type SimpleMockedIO = State MockState

-- |runMockIO simulates IO returning the state after execution
runMockIO :: SimpleMockedIO a -> MockState -> (a, MockState)
runMockIO = runState

-- |Instance example of MockIO for simulation
-- Defines access to inputs, outputs and random numbers.
instance MockIO SimpleMockedIO where
  mPutStr str = do
    st <- get
    let newSt = st { inputs = str:inputs st }
    put newSt
    return ()

  mGetLine = do
    st <- get
    case outputs st of
      [] -> error "MOCK ERROR: Empty stdin"
      (o:ox) -> do
         let newSt = st { outputs = ox }
         put newSt
         return o

  mGetRandomInt min max = do
    st <- get
    case outputInts st of
      [] -> error "MOCK ERROR: Empty random number generator"
      (i:ix) -> do
        if i < min || i > max
         then error "MOCK ERROR: Bad Values in Random Number Generator"
         else do
           let newSt = st { outputInts = ix }
           put newSt
           return i

Conclusion...our Unit Test Works!!!

-- test/DiceSpec.hs
module DiceSpec where

import Test.Hspec (Spec, describe, it, shouldNotBe, shouldThrow, errorCall)

import Dice (rollDice)
import MockIO.Simple

spec :: Spec
spec = do
  describe "Rolling Dice" $ do
    it "Rolls all numbers between 1 and 6" $ do
      let env = MockState [] [] [1,2,3,4,5,6]
      let test :: MockIO m => m [Int]
          test = do
            r1 <- rollDice 6
            r2 <- rollDice 6
            r3 <- rollDice 6
            r4 <- rollDice 6
            r5 <- rollDice 6
            r6 <- rollDice 6
            return [r1, r2, r3, r4, r5, r6]
      let (res,_) = runMockIO test env
      res `shouldBe` [1,2,3,4,5,6]

While this isn't the most glamorous way of solving this problem, it does allow us to swap out `IO` for `State` which is what we needed to make our unit tests work. This solution can be extended to mock network traffic, interactions with a database or pretty much anything you want. It doesn't require a lot of knowledge in advanced topics, having almost everything in here being covered in [Learn You a Haskell][6].

Lastly, if you were worried that now all of your IO functions would be using this `MockIO` type class and not be portable, like if you were wanting to write a public library, don't worry. We can easily expose functions that have the correct `IO` monad type you want.

-- src/Dice.hs
module Dice where

import MockIO
import MockIO.IO

rollDice :: MockIO m => Int -> m Int
rollDice sides = mGetRandomInt 1 sides

rollDiceIO :: Int -> IO Int
rollDiceIO = rollDice

Since `IO` is an instance of `MockIO`, our definition of `rollDiceIO` just refines which `MockIO` instances will be using. Internally you'll use the `MockIO` version of code, along with your unit tests, but only publicly expose the `IO` only version.

>>> import Dice
>>> :t rollDice
rollDice :: MockIO m => Int -> m Int
>>> :t rollDiceIO
rollDiceIO :: Int -> IO Int
>>>

fltkhs [1]

Mock Object [2]

Referential Transparency [3]

Learn You a Haskell: Types and Type Classes[4]

LYaH: For a few Monads more [5]

Learn You a Haskell [6]

Mockio v1 [7]

$ date: 2017-08-21 14:15 $

$ tags: haskell, monads, type classes, testing $

-- CC-BY-SA 4.0 jecxjo 2017-08-21

Comments?

back