As part of a great BDD training session by
Seb Rose, we went about setting up an executable specification environment for our existing C++ codebase. I'm going to talk very briefly about what this is for, and then go into some technical detail about some of the problems we had, and how they've been resolved.
A bit about BDD
BDD - or Behavioural Driven Development - is a method by which you convert non-technical, "behavioural" requirements into executable code, which can then be used to verify the behaviour has been implemented.
Through a process of deliberation with a product owner, tester and technical person, you derive stories, rules, examples and scenarios in a
language understood by everyone, which then becomes an executable specification, which you can use to verify behaviour of the system.
Specflow, Cucumber and Gherkin
The scenarios are parsed. If you're uisng Specflow, the language used is Gherkin, which is parsed by Cucumber. You can read all about here :-
https://github.com/cucumber/cucumber/wiki/Gherkin
The output of this process is C# "feature" stub code for all the Given, When and Then statements found with a feature, with some regular expression powered attribute tags to parse the scenario text into function parameters.
This is cool.
Our setup
We were setting up Specflow in Visual Studio 2013 for use with C++ code that is cross-compiled in IAR Embedded for ARM chips. We do all our unit testing in Visual Studio because frankly, IAR Embedded is terrible. Our test target code is therefore unmanaged C++ code.
This introduces some significant complexities.
We created a fresh C# MSTest project within our existing unit test solution. I've called this "SpecificationTests" in its latest incarnation. We then added Specflow to the plugins for this project (right click the project --> manage NuGet packages).
C# can't talk directly to our unmanaged C++, so we had to create a "shim" layer in C++/CLR.
I was hung up for a while on a good way to organise this underyling "step test code", and Seb suggested by domain entity, so I created a "ScreenSteps" unmanaged C++ project, linked to our REAL code, and a "ScreenStepsShim" C++/CLR project linked to the ScreenSteps project and then added ScreenStepsShim as a reference to the SpecificationTests C#/Specflow project. The "domain entity" in question here, in case it isn't obvious, is the Screen!
The Problems We Had And Their Resolutions
Specflow extension needs to be added to Visual Studio
In addition to adding the plugin to your project, you need to go Tools / Extensions and Updates and search in the online section for SpecFlow, then add that. This was the easiest problem :)
Code generation
Runtime library needs to be the same for projects within the solution.
"Inherit from defaults" seemed to disappear and reappear as an option at will in 2013, so forget about that.
I set everything to multi-threaded debug DLL (Project properties / C++ Code Generation / Runtime Library). I also added the google test project we're using for unit testing into the solution so I could re-build that rather than hard-linking to it.
DLL linkage
Because the unmanaged DLL wasn't exporting the functions, a .lib wasn't being generated and the shim couldn't link to the steps DLL underneath it.
What was missing is DLL export code in the unmanaged C++ header :-
#ifdef SCREENSTEPS_EXPORTS
#define SCREENSTEPS_EXPORT __declspec(dllexport)
#else
#define SCREENSTEPS_EXPORT __declspec(dllimport)
#endif
class SCREENSTEPS_EXPORT ScreenSteps
{
...
};
Now there are functions to be exported, so a .lib is generated (otherwise the linker assumes it is pointless, because there are no functions to call).
Unmanaged DLL not copied by default
AKA "exception thrown by target of the invocation could not load file or assembly or one of its dependencies"...
This was a fun one. When you start trying to call your unmanaged code, it won't work unless the library has been copied into the folder the specflow tests are executing in. The managed library gets copied no problem, but for some reason the unmanaged one does not.
Then, when you try and set the "copy local" option to true, it resets itself to false!!! Wow.
This is apparently a common problem for Microsoft. See here :-
https://connect.microsoft.com/VisualStudio/feedback/details/766064/visual-studio-2012-copy-local-cannot-be-set-via-the-property-pages
They fixed it for a while, and it broke again! So now you have to do it manually in the project file!! Wow.
Adding "<CopyLocal>true</CopyLocal>" to the vcxproj file for ScreenStepsShim seems to work.
<ProjectReference Include="..\ScreenSteps\ScreenSteps.vcxproj">
<Project>...</Project>
<CopyLocal>true</CopyLocal>
</ProjectReference>
Test Explorer hang during test run
This is really annoying. It doesn't happen if you just re-run the tests after changing the feature file, but does happen if you change the code THEN re-run the tests.
When you re-start Visual Studio, you can re-run the tests!
The problem is caused by vstest.exectionengine.x86.exe... I proved this by killing the process and re-trying without restarting Visual Studio. It seems to get excited when you re-build the other libraries and prevents completion of the specflow test build.
Finally I found this :-
http://stackoverflow.com/questions/13497168/vstest-executionengine-x86-exe-not-closing
Add a pre-build event :-
for 64-bit:
taskkill /F /IM vstest.executionengine.exe /FI "MEMUSAGE gt 1"
or for 32-bit:
taskkill /F /IM vstest.executionengine.x86.exe /FI "MEMUSAGE gt 1"
This entirely solves the test runner problem.
std::string to String^ conversion
Unmanaged string to managed C++ String^ (which C# can interpret into a C# string) requires a gcnew in the C++/CLR project, like this :-
string original = someUnmanagedObject.get_SomeText();
String ^converted = gcnew String(original.c_str());
Hopefully this saves other people some pain.
