-
Notifications
You must be signed in to change notification settings - Fork 194
Specs, Testing, and multi color Shoes A Primer
On Fri, Dec 20, 2013 at 5:00 AM, Charles Remes lists@chuckremes.com wrote: I think it might be worth an hour of someone's time to write up some docs (on the wiki?) regarding the recommended way to write specs for Shoes. I sent a PR yesterday with 3 fixes/enhancements that are just begging for proper test coverage but I couldn't figure out how to do it properly. Testing the Shoes classes appears to be impossible since that "backend" is replaced by a mock backend when executing specs. It seems pointless to update the mock backend since it could easily get out of synch with the real class (spec passes because mock is updated but real code fails because the concrete class wasn't updated).
I ran into similar issues when testing the SWT backend. I could mock/stub things out but I'm not sure a passing spec gave me any comfort that the actual class was correct.
I don't like submitting PRs without specs. Right now I feel badly about it and want to fix it myself but I don't have the requisite knowledge or experience with the project structure. Help!
cr
On Fri, Dec 20, 2013 at 5:18 AM, Tobias Pfeiffer tobias.pfeiffer@student.hpi.uni-potsdam.de wrote: Hi there Charles,
thanks for your feedback! This definitely is something we should do and shed more clarity on.
I opened a new issue and hope that I/we will get to some writing about that this year: https://github.com/shoes/shoes4/issues/506
In the meantime: It should help to look at what other specs do. One general speciality is that when you run rake spec at first the specs in specs/shoes/* are executed with the mock backend loaded. They are just there to test pure DSL behavior and maybe that they send some messages to the backend. The mock backend (found in lib/shoes/mock/*) is only meant to provide basic stubs of methods, that the DSL relies on in the GUI. No impls (although there are some, from earlier days)
Then all the tests are executed again, with the SWT backend loaded. E.g. also the tests that already ran with the mocking backend. Now also the specs under spec/swt_shoes/* are executed. Most of the specs for code you wrote should go into this directory as they are backend/SWT specific. In these specs you see a high (sometimes maybe too much) usage of stubs/mocks to try to test the backend classes in isolation. There are efforrs/thoughts to cut down on the stubbing though: We are thinking about using more DSL objects in the SWT specs instead of stubbing them (https://github.com/shoes/shoes4/issues/484) and we got a pull request just about ready to be merged to give all the SWT things a shared context to cut down on duplicated stubbing everywhere (https://github.com/shoes/shoes4/pull/497).
I hope that helped and thanks again for your remarks! =)
Tobi
On Fri, Dec 20, 2013 at 6:04 AM, Peter Fitzgibbons peter.fitzgibbons@gmail.com wrote: HI Tobi, Charles,
Yes, this is a topic I've "explained" a couple of times in email and never got around to creating on wiki... until now. So, my response (below) and this thread is on the wiki : https://github.com/shoes/shoes4/wiki/Specs,-Testing,-and-multi-color-Shoes---A-Primer
When I created the "pluggable" backend, we (Steve and I) discussed how to handle changes to DSL across the multiple backends, especially since the backends could very strongly be completely separate projects. We need a "gold standard" DSL/API -- White Shoes. White Shoes is the "DSL standard" and as you see in the code, it's a Potemkin Village.
So, the Rake routine at this time is : 1. ensure Shoes DSL (as the end-user-developer sees it) complies with the "gold standard" backend - White Shoes. 2. repeat for every backend, exercising the tests on live-gui (when possible). 3. execute the backend's own tests however they are written (the unit-tests of the specific backend).
The complexity here stems from the complexity of the backend-DSL and that the backend cannot be separated (easily) from the hard-dependency upon the computer's graphical UI (SWT for instance). The second level of complexity stems from giving all backend developers (even on separate projects) a known-good target DSL to implement, and follow for ongoing changes.
I hope that clarifies more than it muddies. I wore my Galoshes for the above.
Kindest Regards, SHOES ON! Peter Fitzgibbons