Integration testing of command line tools?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Integration testing of command line tools?

Mark H. Wood
When writing integration tests for command-line tools, is there any
support in Failsafe, jUnit, or elsewhere to fork a process and manage
its standard IO streams?

Or am I over-designing?  Would one typically write such an integration
test rather like a unit test, bypassing the command analyzer and just
calling the appropriate method on an instance created by the test
suite?  Without stubbing or mocking the underlying code, of course,
since it's an integration test.

Is there a better place to ask?

--
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Integration testing of command line tools?

Chris Shellenbarger-2
Have you considered writing an integration test that invoked your command
line tool from the command line?

When you use tools like Failsafe, JUnit, etc, you can know that the process
works in that environment, but the real world execution won't be the same.

While most of your corner cases, logic, etc can be covered by unit tests in
the framework, the final test (does it run on the command line) is
something you should be able to do with a simple shell script (call your
command line, check the output, exit 0 if things match, done).

I think of integration tests as 'testing that everything is hooked up
properly' and so just a couple executions usually can validate this.  I
leave all my edge cases and logic testing to unit tests, usually isolating
at the function level, where they become a lot simpler to test.


---
Chris Shellenbarger
Founder, CloudRepo <https://www.cloudrepo.io>

On Mon, Aug 20, 2018 at 8:59 AM, Mark H. Wood <[hidden email]> wrote:

> When writing integration tests for command-line tools, is there any
> support in Failsafe, jUnit, or elsewhere to fork a process and manage
> its standard IO streams?
>
> Or am I over-designing?  Would one typically write such an integration
> test rather like a unit test, bypassing the command analyzer and just
> calling the appropriate method on an instance created by the test
> suite?  Without stubbing or mocking the underlying code, of course,
> since it's an integration test.
>
> Is there a better place to ask?
>
> --
> Mark H. Wood
> Lead Technology Analyst
>
> University Library
> Indiana University - Purdue University Indianapolis
> 755 W. Michigan Street
> Indianapolis, IN 46202
> 317-274-0749
> www.ulib.iupui.edu
>
Reply | Threaded
Open this post in threaded view
|

Re: Integration testing of command line tools?

Oliver B. Fischer-2
In reply to this post by Mark H. Wood
It depends on the criticality of your command line tools. One
interessting approach is the one by the SpringBoot guys. They run
commands in a Docker container and collect stdout to check the behaviour
of their System V start script.

=>
https://github.com/spring-projects/spring-boot/blob/9d0780c625036ff9cc60ec88c4e4dfabf5a889bc/spring-boot-project/spring-boot-tools/spring-boot-loader-tools/src/test/java/org/springframework/boot/loader/tools/DefaultLaunchScriptTests.java

Oliver

Am 20.08.18 um 15:59 schrieb Mark H. Wood:

> When writing integration tests for command-line tools, is there any
> support in Failsafe, jUnit, or elsewhere to fork a process and manage
> its standard IO streams?
>
> Or am I over-designing?  Would one typically write such an integration
> test rather like a unit test, bypassing the command analyzer and just
> calling the appropriate method on an instance created by the test
> suite?  Without stubbing or mocking the underlying code, of course,
> since it's an integration test.
>
> Is there a better place to ask?
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Integration testing of command line tools?

Thomas Broyer-2
In reply to this post by Mark H. Wood
One approach could be to have a main() overload taking an additional
Environment argument to abstract access to environment variables, system
properties, and standard I/O; and returning an int as the possible
System.exit() value.
That way, you can easily stub the environment for tests.
Your standard main() then simply (and only) call that overload with a
straightforward Environment implementation, and then call System.exit().

Le lun. 20 août 2018 15:59, Mark H. Wood <[hidden email]> a écrit :

> When writing integration tests for command-line tools, is there any
> support in Failsafe, jUnit, or elsewhere to fork a process and manage
> its standard IO streams?
>
> Or am I over-designing?  Would one typically write such an integration
> test rather like a unit test, bypassing the command analyzer and just
> calling the appropriate method on an instance created by the test
> suite?  Without stubbing or mocking the underlying code, of course,
> since it's an integration test.
>
> Is there a better place to ask?
>
> --
> Mark H. Wood
> Lead Technology Analyst
>
> University Library
> Indiana University - Purdue University Indianapolis
> 755 W. Michigan Street
> Indianapolis, IN 46202
> 317-274-0749
> www.ulib.iupui.edu
>