For the longest time, the most commonly-used app on my phone by at least an order of magnitude was Crosswords Classic by Stand Alone, Inc. (They have a newer version that's probably better but every time I've tried it, there's a "who moved my cheese" factor I've never really cared to overcome.) When I had an Android device, it was Shortyz Crosswords. Both excellent and both worth whatever price I paid for them (I've long forgotten).
However, in my advancing years, I've taken to the printed page more and more and in the last year or so, I've been printing crosswords and doing them in pen and paper instead. This was a good move for two reasons: 1) My screen time dropped by more than half, and 2) it opened up several crossword providers that, to my knowledge, aren't available in the apps. For reasons unknown to me, many crossword providers are not making their work available to these apps (e.g. USA Today, Washington Post, and New York Times). In fact, several of them have set up their own payment systems to subscribe to them (e.g. American Values, Matt Gaffney, Elizabeth Gorski).
The upside is that now I have a whole new world of crosswords in an arguably more leisure form. The downside is why you're reading this post: now I have to hunt and peck my way through half a dozen providers, using different delivery mechanisms, rather than have an app just go out and get them. My workflow for this used to be:
Fun fact: Step 3 isn't as onerous as you might think. I spent a good half hour building an Automator action in MacOS to do this until I found out the functionality is built into the right-click menu.
As you can imagine, this virtually reeks of needing automation. So being a good developer, I designed in my head a system that would meet my needs:
(That last one is mostly for USA Today but basically any puzzle provided by uclick.com. They have a special XML format that describes crosswords.)
Let's ignore the cost-benefit of building all this vs. the 10 minutes each week I spend on it manually and chalk it up to: because it's there
I'll skip ahead a bit so I can get to my point. I was a couple of days into this when I remembered a Reddit post I had read recently where someone had automated the download of New York Times crosswords (provided you had a subscription, of course) using a bash script. Then it occurred to me: Do I need all this infrastructure? Azure storage, hosting, logic apps, functions? Could I expand on this bash script from this random, helpful stranger instead?
Skipping ahead a bit further and the answers are, of course: no I don't, and yes I can. The new stack:
I'm using Zapier/Mailparser.io to handle the email providers, C#/XML/XSLT for the uclick providers to transform the XML into HTML (there's very little C#), Chrome to print that HTML to PDF, built-in MacOS to merge the PDFs, and bash to tie it all together.
Here's my new workflow:
... and the only reason I haven't automated #2 is because I haven't looked into the lp
command yet and I like to spot-check the NYT puzzles before I print them.
Now all this is not polished by any definition of the word and it is highly customized to my environment. But by gum, it works. And it still gave me the same rush I've always felt when I've been working at something and it eventually works. Figuring out Zapier's and MailParser's rule systems, re-learning XSLT, seeing HTML files disappearing and working PDFs taking their place. Getting into the minutiae of something that should be easy (like URI-decoding text in XSLT 1.0) but isn't. Not to say, "How can I learn Azure?" but to break a problem down, find a tool, and execute until it works. This is the same routine I used almost 40 years ago when I thought to myself, "I wonder if I can make it say, 'TAKE THAT LOSERS!!!' whenever I score a touchdown" on the old ASCII-based football game in my dad's office computer they let me sit at while waiting for a dentist appointment.
It's likely I'll be the only one who uses this thing, and to be honest, I hope I am because the idea of supporting it gives me the shakes. My original design was certainly more ambitious and might be easier to turn into a conference talk. But it's overkill. Maybe there's a market for applications that collect and manage crossword puzzle PDFs; I'm past the point where I want to build an application that meets that need first and my personal need second.
Kyle the Unsolveable
]]>In the article, the author describes their Mr. Destiny moment where they wish they had gone into management rather than sticking with being "just" a programmer. Thus perpetuating the myth in our industry that you aren't worth anything unless you Change The World™.
To be fair, the author is careful not to be prescriptive. It's very much "these are my regrets and thoughts" and not "you should do this in the same situation". I'm thankful for that and I'll pay the same courtesy. These are my thoughts as they apply to me personally. If you can relate to it in any way, that's not my fault.
Twenty years ago, I suppose I was at my own crossroads though I didn't really recognize it as such until I thought about it just now. Ever since I was in grade 8, I was going to be an actuary. At the time, I was told "you have to be good at math and you'll make a lot of money". What else do you need to hear when you're in grade 8?
So I set about my goal and got a degree, a Bachelor of Commerce I believe. I wasn't really sure what to make of the Faculty of Management, what with their wine and cheeses and people coming to class in suits and "networking" but whatever, I did the work, finished my studies, and started interviewing like you were supposed to.
At some point in that first six months of interviewing, I clued in to something: programming, which I had been doing for about 10 years by that point as a hobby, was a career choice! Was it a lucrative one? Was it better than being an actuary? Who cares, it was fun and people got paid to do it! So I went back to school. That is, to my ageing memory, the sum total of the thought I put into it. (This will be a recurring theme throughout this post.)
So I did it. I went back to school, got the degree, and started working. My first job was in the corporate world, an oil and gas company. My next, with a startup where I interviewed badly and they hired me anyway. The third was with a consulting company. Since then: contracting, my own startup, and employeehood in some order. Along the way, I've learned, to varying degrees, VB6, classic ASP, .NET, RPG, Livelink, Sharepoint, JavaScript, Ruby, Java, SQL Server, Azure, Google Web Toolkit, Docker, CI, CD, CQRS, CORS, CSS and more acronymed software methodologies and techniques than I care to put thought into remembering admit. I've blogged (clearly), co-written a book, spoken at conferences, and created a user group that was the first of its kind in the country. (It lasted less than a year.)
While you're free to copy and paste all this into my obituary, there's a reason I list it out. I have never put much thought into my career and I don't have any intrinsic itch I've been meaning to scratch. How did I choose when and where to do all of these things? The opportunities came up and I said yes. Again, not much more thought goes into it than that.
I have no regrets (including the book). Certainly not on a macro level. Should I have pursued a career as an actuary which, in all likelihood, would have ended up more lucrative financially? Was it a waste of time learning Livelink? Should I have gone into a managerial role?
Does it matter?
I am where I am now because of the choices I've made. I've often joked that the main criteria I use to choose my contracts/positions is whether I think it'll be fun. Not whether it was in a hot technology or a dynamic industry. I'm too lazy to figure out what constitutes either of those.
The net result of those criteria is that I can look back favourably on a career (and I use that term loosely) as "just a programmer". One that I have no intention of leaving because why would I?
I am, and I can't emphasize this enough, having the time of my life.
Now to be fair, I've always had the power to see things through a pair of glasses a supernatural shade of rose. But consider the language from the post I mentioned earlier:
I can only imagine how in demand I could have been.
My sister has 10X the assets I have.
And my favourite one:
And today I am still just a programmer. Who’s the weenie now?
To me, this speaks of a chronic problem of chasing the wrong thing. What you're supposed to do versus what you want to do. And I've often wondered, if you had taken that other path, would you still have no regrets?
The world is filled with people who want to leave their mark, some of whom actually will. And I think it's great that these people exist. We need them. For my part, I see no point in feeling guilt or regret doing something I love doing in an industry that is privileged enough to provide me with a good living doing it.
]]>In addition to the slides, you can find the source code for the presentation here. There's also a blog post that expands a little on the ideas.
The two sample apps I showed during the presentation are:
For the first, I showed how to connect the app to PostgreSQL which can be running either locally or in Docker. For the second, I showed how to create a container for a .NET Core application running in Linux. If you want to play with this container, you can get it here on Docker Hub.
In addition to the presentations, Donald and I (but mostly me), with the help of the Winnipeg .NET User Group, hosted a fishbowl discussion and an evening of drinks and food the night before the conference courtesy of Particular Software. During the talk, we covered ways to build a solid team, the importance of company culture, and good ways to vet people. It was a great chat and it was nice to see so many people weigh in with their experience.
It's hard to overstate the value this "big little conference" has for me. It has all the benefits of a larger conference while still being accessible to both the attendees and speakers. D'Arcy Lussier is a force to be reckoned with in the conference circuit.
Many thanks to the Winnipeg .NET User Group for their help with the fishbowl event.
Finally, a big thank you to those of you who came to the the conference itself and continue making it great.
]]>I've talked on this subject before and it's something I often have in the back of my mind when reading comments. It's come to the forefront recently with some conversations we've had at Western Devs, which led to our most recent podcast. I wasn't able to attend so here I am.
There are certain phrases you see in comments that automatically seem to devolve a discussion. They include:
Ultimately, all of these phrases can be summarized as follows:
In my younger years, I could laugh this off amiably and say "Oh this wacky world we live in". But I'm turning 44 in a couple of days and it's time to start practicing my crotchety, even if it means complaining about people being crotchety.
So to that end: I'm asking, nay, begging you to avoid these and similar phrases. This is for your benefit as much as the reader's. These phrases don't make you sound smart. Once you use them, it's very unlikely anyone involved will feel better about themselves, let alone engage in any form of meaningful discussion. Even if you have a valid point, who wants to be talked down to like that? Have you completely forgot what it's like to learn?
Now I know (from experience) it's hard to fight one's innate sense of superiority and the overwhelming desire to make it rain down on the unwashed heathen. So take it in steps. After typing your comment, remove all instances of "just" (except when just means "recently" or "fair", of course). The same probably goes for "simply". It has more of a condescending tone than a dismissive one. "Actually" is borderline. Rule of thumb: Don't start a sentence with it.
Once you have that little nervous tic under control, it's time to remove the negatives. Here's a handy replacement guide to get you started:
Original phrase | Replacement |
---|---|
"Can't you" | "Can you" |
"Why don't you" | "Can you" |
"Sorry but" | no replacement; delete the phrase |
"It's amazing that..." | delete your entire comment and have a dandelion break |
See the difference? Instead of saying Sweet Zombie Jayzus, you must be the stupidest person on the planet for doing it this way, you've changed the tone to Have you considered this alternative? In both instances, you've made your superior knowledge known but in the second, it's more likely to get acknowledged. More importantly, you're less likely to look like an idiot when the response is: I did consider that avenue and here are legitimate reasons why I decided to go a different route.
To be fair, sometimes the author of the work you're commenting on needs to be knocked down a peg or two themselves. I have yet to meet one of these people who respond well to constructive criticism critique, let alone the destructive type I'm talking about here. Generally, I find they feel the need to cultivate an antagonistic personality but in my experience, they usually don't have the black turtlenecks to pull it off. Usually, it ends up backfiring and their dismissive comments become too easy to dismiss over time.
Kyle the Inclusive
]]>Jekyll and Hexo handle excerpts very differently. In Jekyll, you can specify an excerpt in the front matter. If one isn't provided, it will use the first few characters of the post itself as the excerpt.
In Hexo, the excerpt is set in one and only one way. You need to add <!--more-->
somewhere in the post. Everything before that is considered the excerpt. If you don't have that tag in your post, no excerpt.
Luckily, Amir had some foresight and added code to our Jekyll site to support <!--more-->
. Unluckily, we rarely used it. Some of our posts had excerpts, particularly the podcasts. But mostly, we relied on Jekyll to parse the post for the excerpt.
In the end, we did two things to remedy this. First we added the https://github.com/lalunamel/hexo-front-matter-excerpt package. This fixed all the posts that already had an excerpt
in the front matter. For the rest, we went through all the remaining posts and added <!--more-->
after the first paragraph. Ever the optimists, we also submitted a pull request for the package to make it behave more like Jekyll.
A code block in Jekyll looks like this
1 | {% highlight html%} |
And in Hexo, like this:
1 | {% codeblock lang:html %} |
A series of "Find and replace in files" actions took care of this. We also found a syntax.styl
file to our liking. Not sure if Jekyll has built-in support for syntax highlighting. I suspect not and that it was automatically included with the theme we chose for Jekyll.
Jekyll uses Kramdown for parsing Markdown. One of the features it adds is the ability to specify a class on your elements like so.
1 | {: .pull-right } |
This will add the pull-right
CSS class to the image and, as the name suggests, this will float the image on the right.
As far as I know, there's no Kramdown support in NodeJS, likely because Kramdown is a Ruby gem. So we had to modify each instance of this syntax manually. For images, we converted to the Hexo img tag plugin like so:
1 | {% img pull-right "http://my/image/reference" "My image"} |
There are a few tag plugins in Hexo all ported from Octopress. Which likely means we could have used them in our Jekyll site but not much point in testing that out now...
For other instances of this kramdown-specific syntax that were applied to other elements, we just dropped down to native HTML in the Markdown:
1 | <div class="notice">This is a notice</div> |
We changed the URL structure for our podcasts early on and to keep any links pointing to the existing ones valid, we used the jekyll-redirect-from gem to maintain the old link. We find a suitable replacement for it in Hexo. BUT that suitable replacement's official package on NPM doesn't support Hexo 3 at the time of writing so we had to reference it in our package.json
file like so:
1 | "hexo-generator-alias": "git+https://github.com/hexojs/hexo-generator-alias.git" |
Bear in mind that this is a static site. So both of these plugins, the Jekyll and the Hexo one, handle redirects by generating an HTML page at the alias location with a redirect embedded in it:
1 |
|
This isn't quite ideal for SEO purposes but it is the recommended approach if you can't do a server-side redirect.
This issue was probably the source of most of our trouble. Our permalink URL is of the form /:category/:title
. For example: /docker/yet-another-docker-intro/
. The issue is that the vast majority of our posts are uncategorized. And Jekyll and Hexo each handle uncategorized posts very differently in their permalinks.
In the front matter for Hexo, you can define a default category which all posts will use if a category is not assigned. So if you set the default category to "moo", your permalink URL will be: /moo/a-discussion-on-knockout/
.
In Jekyll, an uncategorized post appears at the root of the site. Like this: http://www.westerndevs.com/Migrating-from-Jekyll-to-Hexo-Part-2
. But setting the default category to an empty string or to /
, led to fully-qualified URLs that look like this: //Migrating-from-Jekyll-to-Hexo-Part-2
. I believe this is a bug in the permalink generator. It's generating the permalink as /category/title and when there is no category, it's not going to treat it any differently.
Our initial solution to this was to stick with a default category of uncategorized
but to alias each post to the root so that existing URLs would still work but would redirect to the new one. Alas, we have Disqus comments to deal with and those are tied to a specific URL. We could have migrated them but we also would have had to contend with sucky looking URLs that have uncategorized
in them.
Our current solution: fork Hexo. In our fork, we add some handling when generating the URL for a post so that if it starts with a //, we trim one of them. Not the most elegant solution but workable for now. And if that's not hacky enough for you, check out the solution for...
After deployment, we discovered none of the existing Disqus comments were showing on our posts. The Disqus script was working because you could add new comments and it would show "Also on Western Devs" comments. Just no existing comments.
The culprit was, again, the permalink. Because of the leading //, Disqus thought the URL for our posts was (as an example): http://www.westerndevs.com//a-discussion-on-knockout. That double slash was enough to confuse it into not showing comments.
Our interim solution is to include this in our post template:
1 | var disqus_url = '<%= page.permalink %>'.replace(".com//", ".com/"); |
That right there is some quality coding...
This issue is more of a warning not to over-think things. Our feed is served up at http://www.westerndevs.com/feed. In Jekyll, we created a feed.xml
file and assumed that it did some magic to generate a feed
file. And we had a hell of a time trying to get Hexo to generate that same file without the .xml extension. Even to the point where we created our own package that still didn't quite work.
Then someone realized that Jekyll was not actually generating a feed
file, just a feed.xml
file. Whatever GitHub Pages runs on allows you to access XML files without the extension. Jekyll's development server knows this. Hexo's doesn't. So when we run locally, we can't access our various XML files (for our RSS feed, our iTunes feed, and our sitemap) without specifying the extension. But on the deployed site, they work fine.
There was a bug in the default Markdown renderer that caused rendering issues when you had links within headers. The bug has since been fixed but I'm including it here mostly to reiterate that you aren't locked into anything with Hexo. While we were converting, the bug was still active and all we did to fix it was to switch to a different renderer.
One philosophical point: One of the reasons for this bug is that the default renderer will automatically add bookmarks for every heading. This is a design decision that I'm not convinced has a lot of value though it's not a position I'll defend strongly. The renderer we've switched to includes this as an option you have to explicitly turn on which I think is a better way to go.
Since GitHub Pages runs Jekyll, it's understandable that deployment with Jekyll is relatively easy. We did need to wrap it in a Rake task to accommodate working locally with a different _config.yml
file than what is used in production but that wasn't hard to do.
For Hexo, our process is still mostly manual and has to be done locally rather than on our CI server (Travis). We generate the site, copy the static files to a folder, then check those files into the gh-pages
branch. I imagine it won't be hard to get our Travis process adapted to this but at present, we haven't got around to it yet. Just keep in mind, it won't be quite as easy as Jekyll.
I forgot to answer this in the last post. As I mentioned, we're enjoying Hexo and it's nice having everyone excited about blogging again. For the Western Devs as a group, Hexo is the better choice.
But personally, I like Jekyll a little better. It feels more polished and doesn't require you to fork the product to get what you want. Typing this up, I was looking for a way to disable line numbers for individual codeblocks but I've come up short. Plus there's a larger and more comprehensive community behind it. That said, I am really enjoying the quick generation time in Hexo.
I believe that covers the major gotchas we encountered in our conversion. It glosses over a few things, like whether to stick with Stylus as the default CSS pre-processor or move to SASS. Or whether to use Jade as the templating language rather than the default, EJS. Those questions are quite a bit more subjective and I wanted to keep this discussion limited to the technical hurdles we encountered.
But if you run into an issue not mentioned here, add a comment and I will update the post.
]]>Regardless, here we are and it's already been a positive move overall, not least because the move to Node means more of us are available to help with the maintenance of the site. But it wasn't without it's challenges. So I'm going to outline the major ones we faced here in the hopes that it will help you make your decision more informed than ours was.
To preface this, note that I'm new to Node and in fact, this is my first real project with it. That said, I'm no expert in Ruby either, which is what Jekyll is written in. And the short version of my first impressions is: Jekyll feels more like a real product but I had an easier time customizing Hexo once I dug into it. Here's the longer version
You'll run into this very quickly. Documentation for Hexo is decent but incomplete. And once you start Googling, you'll discover many of the resources are in Chinese. I found very quickly that there is posts
collection and that each post has a categories
collection. But as to what these objects look like, I couldn't tell. They aren't arrays. And you can't JSON.stringify
them because they have circular references in them. util.inspect
works but it's not available everywhere.
By default, Hexo doesn't support multiple authors. Neither does Jekyll, mind you, but we found a pretty complete theme that does. In Hexo, there's a decent package that gets you partway there. It lets you specify an author ID on a post and it will attach a bunch of information to it. But you can't, for example, get a full list of authors to list on a Who We Are page. So we created a separate data file for the authors. But we also haven't figured out how to use that file to generate a .json file to use for the Featured section on the home page. So at the moment, we have author information in three places. Our temporary solution is to disallow anyone from joining or leaving Western Devs.
If you go with Hexo and choose an existing themes, you won't run into the same issues we did. Out of the box, it has good support for posts, categories, pagination, even things like tags and aliases with the right plugins.
But we started from a design and were migrating from an existing site with existing URLs and had to make it work. I've mentioned the challenge of multiple authors already. Another one: maintaining our URLs. Most of our posts aren't categorized. In Jekyll, that means they show up at the root of the site. In Hexo, that's not possible. At least at the moment and I suspect this is a bug. We eventually had to fork Hexo itself to maintain our existing URLs.
Another challenge: excerpts. In Jekyll, excerpts work like this: Check the front matter for an excerpt
. If one doesn't exist, take the first few characters from the post. In Hexo, excerpts are empty by default. If you add a <!--more-->
tag in your post, everything before that is considered an excerpt. If you specify an excerpt
in your front matter, it's ignored because there is already an excerpt
property on your posts.
Luckily, there's a plugin to address the last point. But it still didn't address the issue of all our posts without an excerpt where we relied solely on the contents of the post.
So if you're looking to veer from the scripted path, be prepared. More on this later in the "good parts" section.
This is more a culmination of the previous issues. It just feels like Hexo is a work-in-progress whereas Jekyll feels more like a finished product. There's a strong community behind Jekyll and plenty of help. Hexo still has bugs that suggest it's just not used much in the wild. Like rendering headers with links in them. It makes the learning process a bit challenging because with Jekyll, if something didn't work, I'd think I'm obviously doing something wrong. With Hexo, it's I might be doing something wrong or there might be a bug.
I said earlier that the move to Hexo was positive overall and not just because I'm optimistic by nature. There are two key benefits we've gained just in the last two weeks.
Hexo is fast, plain and simple. Our old Jekyll site took six seconds to generate. Doesn't sound like much but when you're working on a feature or tweaking a post, then saving, then refreshing, then rinsing, then repeating, that six seconds adds up fast. In Hexo, a full site generation takes three seconds. But more importantly, it is smart enough to do incremental updates while you're working on it. So if you run hexo server
, then see a mistake in your post, you can save it, and the change will be reflected almost instantly. In fact, it's usually done by the time I've switched back to the browser.
We had logistical challenges with Jekyll. To the point where we had two methods for Windows users that wanted to contribute (i.e. add a post). One involved a Docker image and the other Azure ARM. Neither was ideal as they took between seconds and minutes to refresh if you made changes. Granted, both methods furthered our collective knowledge in both Docker and Azure but they both kinda sucked for productivity.
That meant that realistically, only the Mac users really contributed to the maintenance of the site. And our Docker/Azure ARM processes were largely ignored as we would generally just test in production. I.e. create a post, check it in, wait for the site to deploy, make necessary changes, etc, etc.
With the switch to Hexo, we've had no fewer than five contributors to the site's maintenance already. Hexo just works on Windows. And on Mac. Best of both worlds.
This is listed under the challenges but ever the optimist, I'm including it here as well. We've had to make some customizations for our site, including forking Hexo itself. And for me personally, once I got past the why isn't this working the way I want? stage, it's been a ton of fun. It's crazy simple to muck around in the node modules to try stuff out. And just as simple to fork something and reference it in your project when the need arises. I mentioned an earlier issue rendering links in headers. No problem, we just swapped out the markdown renderer for another one. And if that doesn't work, we'll tweak something until it does.
I want to talk more on specific conversion issues we ran into as a guide to those following in our footsteps. But there are enough of them to warrant a follow up post without all this pre-amble. For now, we're all feeling the love for Hexo. So much so that no less than three other Western Devs are in the process of converting their personal blogs to it.
]]>I was asked recently for some advice on generating data for use with integration and UI tests. I already have some ideas but asked the rest of the Western Devs for some elucidation. My tl;dr version is the same as what I mentioned in our discussion on UI testing: it's hard. But manageable. Probably.
The solution needs to balance a few factors:
The two options we discussed both assume the first factor to be immutable. That means you either clean up after yourself when the test is finished or you wipe out the database and start from scratch with each test. Cleaning up after yourself might be faster but has more moving parts. Cleaning up might mean different things depending on which step you're in if the test fails.
So given that we will likely re-create the database from scratch before each and every test, there are two options. My current favourite solution is a hybrid of the two.
In this option, you have a pre-configured database. Maybe it's a SQL Server .bak file that you restore before each test. Maybe it's a GenerateDatabase
method that you execute. I've done the latter on a Google App Engine project, and it works reasonably well from an implementation perspective. We had a class for each domain aggregate and used dependency injection. So adding a new test customer to accommodate a new scenario was fairly simple. There are a number of other ways you can do it, some of which Simon touched on in his post.
We also had it set up so that we could create only the customer we needed for that particular test if we needed to. That way, we could use a step like Given I'm logged into 'Christmas Town'
and it would set up only that data.
There are some drawbacks to this approach. You still need to create a new class for a new customer if you need to do something out of the ordinary. And if you need to do something only slightly out of the ordinary, there's a strong tendency to use an existing customer and tweak its data ever so slightly to fit your test's needs, other tests be damned. With these tests falling firmly in the long-running category, you don't always find out the effects of this until much later.
Another drawback: it's not obvious in the test exactly what data you need for that specific test. You can accommodate this somewhat just with a naming convention. For example, Given I'm logged into a company from India
, if you're testing how the app works with rupees. But that's not always practical. Which leads us to the second option.
Here, your API contains steps to fully configure your database exactly the way you want. For example:
1 | Given I have a company named "Christmas Town" owned by "Jack Skellington" |
You can probably see the major drawback already. This can become very verbose. But on the other hand, you have the advantage of seeing exactly what data is included which is helpful when debugging. If your test data is wrong, you don't need to go mucking about in your source code to fix it. Just update the test and you're done.
Also note the lack of specifics in the steps. Whenever possible, I like to be very vague when setting up my test data. If you have a good framework for generating test data, this isn't hard to do. And it helps uncover issues you may not account for using hard-coded data (as anyone named D'Arcy O'Toole can probably tell you).
Loading up your data with a granular API isn't realistic which is why I like the hybrid solution. By default, you pre-load your database with some common data, like lookup tables with lists of countries, currencies, product categories, etc. Stuff that needs to be in place for the majority of your tests.
After that, your API doesn't need to be that granular. You can use something like Given I have a basic company
which will create the company, add an owner and maybe some products and use that to test the process for creating an order. Under the hood, it will probably use the specific steps.
One reason I like this approach: it hides only the details you don't care about. When you say Given I have a basic company and I change the name to "Rick's Place"
, that tells me, "I don't care how the company is set up but the company name is important". Very useful to help narrow the focus of the test when you're reading it.
This approach will understandably lead to a whole bunch of different methods for creating data of various sizes and coarseness. And for that you'll need to...
Regardless of your method, maintaining your test data will require constant vigilance. In my experience, there is a tremendous urge to take shortcuts when it comes to test data. You'll re-use a test company that doesn't quite fit your scenario. You'll alter your test to fit the data rather than the other way around. You'll duplicate a data setup step because your API isn't discoverable.
Make no mistake, maintaining test data is work. It should be treated with the same respect and care as the rest of your code. Possibly more so since the underlying code (in whatever form it takes) technically won't be tested. Shortcuts and bad practices should not be tolerated and let go because "it's just test data". Fight the urge to let things slide. Call it out as soon as you see it. Refactor mercilessly once you see opportunities to do so.
Don't be afraid to flip over a table or two to get your point across.
-- Kyle the Unmaintainable
]]>The sample application is just a plain ol' .NET application. It is meant to showcase different ways of doing things. One of those things is data access. You can configure the app to access the data from SQL storage, Azure table storage, or in-memory. By default, it uses the in-memory option so you can clone the app and launch it immediately just to see how it works.
Quick summary: Calgary, Alberta hosts an annual event called the Calgary Stampede. One of the highlights of the 10-ish day event is the pancake breakfast, whereby dozens/hundreds of businesses offer up pancakes to people who want to eat like the pioneers did, assuming the pioneers had pancake grills the size of an Olympic swimming pool.
The sample app gives you a way to enter these pancake breakfast events and each day, will show that day's breakfasts on a map. There's also a recipe section to share pancake recipes but we won't be using that here.
To work with Docker we need to set the app up to use a data access mechanism that will work on Docker. The sample app supports Postgres so that will be our database of choice. Our first step is to get the app up and running locally with Postgres without Docker. So, assuming you have Postgres installed, find the ContainerBuilder.cs
file in the PancakeProwler.Web
project. In this file, comment out the following near the top of the file:
1 | // Uncomment for InMemory Storage |
And uncomment the following later on:
1 | // Uncomment for PostgreSQL storage |
This configures the application to use Postgres. You'll also need to do a couple of more tasks:
Postgres
connection string in the web project's web.config
to match the username and database you createdThe first two steps can be accomplished with the following script in Postgres:
1 | CREATE DATABASE "Pancakes"; |
Save this to a file. Change the username/password if you like but be aware that the sample app has these values hard-wired into the connection string. Then execute the following from the command line:
psql -U postgres -a -f "C:\path\to\sqlfile.sql"
At this point, you can launch the application and create events that will show up on the map. If you changed the username and/or password, you'll need to update the Postgres connection string first.
You might have noticed that you didn't create any tables yet but the app still works. The sample is helpful in this regard because all you need is a database. If the tables aren't there yet, they will be created the first time you launch the app.
Note: recipes rely on having a search provider configured. We won't cover that here but I hope to come back to it in the future.
Next, we'll switch things up so you can run this against Postgres running in a Docker container.
I'm going to give away the ending here and say that there is no magic. Literally, all we're doing in this section is installing Postgres on another "machine" and connecting to it. The commands to execute this are just a little less click-y and more type-y.
The first step, of course, is installing Docker. At the time of writing, this means installing Docker Machine.
With Docker Machine installed, launch the Docker Quickstart Terminal and wait until you see an ASCII whale:
If this is your first time running Docker, just know that a lightweight Linux virtual machine has been launched in VirtualBox on your machine. Check your Start screen and you'll see VirtualBox if you want to investigate it but the docker-machine
command will let you interact with it for many things. For example:
docker-machine ip default
This will give you the IP address of the default virtual machine, which is the one created when you first launched the Docker terminal. Make a note of this IP address and update the Postgres connection string in your web.config to point to it. You can leave the username and password the same:
1 | <add name="Postgres" connectionString="server=192.168.99.100;user id=Matt;password=moo;database=Pancakes" providerName="Npgsql" /> |
Now we're ready to launch the container:
docker run --name my-postgres -e POSTGRES_PASSWORD=moo -p 5432:5432 -d postgres`
Breaking this down:
docker run | Runs a docker container from an image |
--name my-postgres | The name we give the container to make it easier for us to work with. If you leave this off, Docker will assign a relatively easy-to-remember name like "floral-academy" or "crazy-einstein". You also get a less easy-to-remember identifier which works just as well but is...less...easy-to-remember |
-e POSTGRES_PASSWORD=moo | The -e flag passes an environment variable to the container. In this case, we're setting the password of the default postgres user |
-p 5432:5432 | Publishes a port from the container to the host. Postgres runs on port 5432 by default so we publish this port so we can interact with Postgres directly from the host |
-d | Run the container in the background. Without this, the command will sit there waiting for you to kill it manually |
postgres | The name of the image you are creating the container from. We're using the official postgres image from Docker Hub. |
If this is the first time you've launched Postgres in Docker, it will take a few seconds at least, possibly even a few minutes. It's downloading the Postgres image from Docker Hub and storing it locally. This happens only the first time for a particular image. Every subsequent postgres container you create will use this local image.
Now we have a Postgres container running. Just like with the local version, we need to create a user and a database. We can use the same script as above and a similar command:
psql -h 192.168.99.100 -U postgres -a -f "C:\path\to\sqlfile.sql"
The only difference is the addition of -h 192.168.99.100
. You should use whatever IP address you got above from the docker-machine ip default
command here. For me, the IP address was 192.168.99.100.
With the database and user created, and your web.config updated, we'll need to stop the application in Visual Studio and re-run it. The reason for this is that the application won't recognize that we've changed database so we need to "reboot" it to trigger the process for creating the initial table structure.
Once the application has been restarted, you can now create pancake breakfast events and they will be stored in your Docker container rather than locally. You can even launch pgAdmin (the Postgres admin tool) and connect to the database in your Docker container and work with it like you would any other remote database.
From here, where you go is up to you. The sample application can be configured to use Elastic Search for the recipes. You could start an Elastic Search container and configure the app to search against that container. The principle is the same as with Postgres. Make sure you open both ports 9200 and 9300 and update the ElasticSearchBaseUri
entry in web.config
. The command I used in the presentation was:
docker run --name elastic -p 9200:9200 -p 9300:9300 -d elasticsearch
I also highly recommend Nigel Poulton's Docker Deep Dive course on Pluralsight. You'll need access to Linux either natively or in a VM but it's a great course.
There are also a number of posts right here on Western Devs, including an intro to Docker for OSX, tips on running Docker on Windows 10, and a summary or two on a discussion we had on it internally.
Other than that, Docker is great for experimentation. Postgres and Elastic Search are both available pre-configured in Docker on Azure. If you have access to Azure, you could spin up a Linux VM with either of them and try to use that with your application. Or look into Docker Compose and try to create a container with both.
For my part, I'm hoping to convert the sample application to ASP.NET 5 and see if I can get it running in a Windows Server Container. I've been saying that for a couple of months but I'm putting it on the internet in an effort to make it true.
]]>After posting giddily on Docker in the Windows world recently, Microsoft released Windows Server 2016 Technical Preview 3 with container support. I've had a chance to play with it a little so let's see where this goes...
Like movie previews, this is equal parts exciting and frustrating. Exciting because you get a teaser of things to come. Frustrating because you just want it to work now. And extra frustration points for various technical issues I've run into that, I hope, are due to the "technical preview" label.
For example, installing container support into an existing VM is mind-numbingly slow. Kudos to the team for making it easy to install but at the point where you run ContainerSetup.ps1, be prepared to wait for, by my watch, at least 45 minutes without any visual indication that something is happening. The only reason I knew something was happening is because I saw the size of the VM go up (slowly) on my host hard drive. This is on a 70Mbps internet connection so I don't think this can be attributed to "island problems" either.
I've heard tell of issues setting up container support in a Hyper-V VM as well. That's second-hand info as I'm using Fusion on a Mac rather than Hyper-V. If you run into problems setting it up on Hyper-V, consider switching to the instructions for setting up containers on non-Hyper-V VMs instead (no longer available).
There's also the Azure option. Microsoft was gracious enough to provide an Azure image for Windows Server 2016 pre-configured with container support. This works well if you're on Azure and I was able to run the nginx tutorial on it with no issues. I had less success with the IIS 10 tutorial even locally. I could get it running but was not able to create a new image based on the container I had.
Technical issues aside, I haven't been this excited about technology in Windows since...ASP.NET MVC, I guess, if my tag cloud is to be believed. And since this is a technical preview designed to garner feedback, here's what I want to see in the Windows container world
I love that I can use the Docker client to work with Windows containers. I can leverage what I've already learned with Docker in Linux. But I also love that I can spin up containers with PowerShell so I don't need to mix technologies in a continuous integration/continuous deployment environment if I already have PowerShell scripts set up for other aspects of my process.
I can't take credit for this. I've been talking with Gabriel Schenker about containers a lot lately and it was he who suggested they need to have support for .NET 4, .NET 3.5, and even .NET 2.0. It makes sense though. There are a lot of .NET apps out there and it would be a shame if they couldn't take advantage of containers.
Docker Machine is great for getting up and running fast on a local Windows VM. To fully take advantage of containers, devs need to be able to work with them locally with no friction, whether that means a Windows Container version of Docker Machine or the ability to work with containers natively in Windows 10.
At Western Devs, we have a PowerShell script that will spin up a new Azure Linux virtual machine, install docker, create a container, and run our website on it. It goes without saying (even though I'm saying it) that I'd like to do the same with Windows containers.
I'd like to wean myself off VMs a little. I picture a world where I have one base VM and I use various containers for the different pieces of the app I'm working on. E.g. A SQL Server container, an IIS container, an ElasticSearch container, possibly even a Visual Studio container. I pick and choose which containers I need to build up my dev environment and use just one (or a small handful) of VMs.
In the meantime, I'm excited enough about Windows containers that I hope to incorporate a small demo with them in my talk at MeasureUP in a few scant weeks so if you're in the Austin area, come on by to see it.
It is a glorious world ahead in this space and it puts a smile on this hillbilly's face to see it unfold.
Kyle the Barely Contained
]]>Western Devs uses Jekyll. Someone suggested it, I tried it, it worked well, decision was done. Except that it doesn't work well on Windows. It's not officially supported on the platform and while there's a good guide on getting it running, we haven't been able to do so ourselves. Some issue with a gem we're using and Nokogiri and lib2xml and some such nonsense.
So in an effort to streamline things, Amir Barylko create a Docker image. It's based on the Ruby base image (version 2.2). After grabbing the base image, it will:
bundle install
With this in place, Windows users can run the website locally without having to install Ruby, Python, or Jekyll. The command to launch the container is:
docker run -t -p 4000:4000 -v //c/path/to/code:/root/jekyll abarylko/western-devs:v1 sh -c 'bundle install && rake serve'
This will:
abarylko/western-devs:v1
imagebundle install && rake serve
to update gems and launch Jekyll in the containerTo make this work 100%, you also need to expose port 4000 in VirtualBox so that it's visible from the VM to the host. Also, I've had trouble getting a container working with my local source located anywhere except C:\Users\mysuername. There's a permission issue somewhere in there where the container appears to successfully map the drive but can't actually see the contents of the folder. This manifests itself in an error message that says Gemfile not found
.
Now, Windows users can navigate to localhost:4000 and see the site running locally. Furthermore, they can add and make changes to their posts, save them, and the changes will get reflected in the browser. Eventually, that is. I've noticed a 10-15 second delay between the time you press Save to the time when the changes actually get reflected. Haven't determined a root cause for this yet. Maybe we just need to soup up the VM.
So far, this has been working reasonably well for us. To the point, where fellow Western Dev, Dylan Smith has automated the deployment of the image to Azure via a Powershell script. That will be the subject of a separate post. Which will give me time to figure out how the thing works.
]]>I created a VM in Fusion for the server which went pretty smoothly. When presented with a list of operating systems, I selected Windows Server 2012 and it installed fine from the ISO file. After that, I started on the documentation and at the step where you run the ContainerSetup.ps1 powershell script, I got hit with an error:
New-NetNat : No matching interface was found for prefix (null).At C:\ContainerSetup.ps1:247 char:5New-NetNat -Name ContainerNAT -InternalIPInterfaceAddressPrefix $ ...CategoryInfo : NotSpecified (MSFT_NetNat:root/StandardCimv2/MSFT_NetNat) [New-NetNat], CimExceptionFullyQualifiedErrorId : Windows System Error 1169,New-NetNat
This tripped me up for a while for a few reasons:
The answer, which came to me in a rubber ducking episode as I typed on a question on the forums, was in the VM's network configuration in VMWare Fusion:
![Network settings](http://i.imgur.com/nzfzr4X.png = 200x)
By default, VMs get created with the "Share with my Mac" setting. Changing this to Autodetect allowed the script to continue.
By that time, I had discovered that an Azure VM image already exists for Server 2016 Tech Preview 3 and even with the Containers Feature already enabled. So that's as far as I ventured with the VM.
]]>In the meantime, I have to temper my enthusiasm for the time being because Docker is still a Linux-only concern. Yes, you can run Docker containers on Windows but only Linux-based ones. So no SQL Server and no IIS.
But you can't stop a hillbilly from dreaming of a world of containers. So with a grand assumption that you know what Docker is roughly all about, here's what this coder of the earth meditates on, Docker-wise, before going to sleep.
Microservices are a hot topic these days. We've talked about them at Western Devs already and Donald Belcham has a good and active list of resources. Docker is an eerily natural fit for microservices so much so that one might think it was created specifically to facilitate the architecture. You can package your entire service into a container and deploy it as a single package to your production server.
I don't think you can understate the importance of a technology like Docker when it comes to microservices. Containers are so lightweight and portable, you just naturally gravitate to the pattern through normal use of containers. I can see a time in the near future where it's almost negligent not to use microservices with Docker. At least in the Windows world. This might already be the case in Linux.
Ah, the crutch of the developer and the bane of DevOps. You set it up so nicely on your machine, with all your undocumented config entries and custom permissions and the fladnoogles and the whaztrubbets and everything else required to get everything perfectly balanced. Then you get your first bug from QA: can't log in.
But what if you could test your deployment on the exact same image that you deployed to? Furthermore, what if, when a bug came in that you can't reproduce locally, you could download the exact container where it was occurring? NO MORE EXCUSES, THAT'S WHAT!
On one project, we had a suite of UI tests which took nigh-on eight hours in TeamCity. We optimized as much as we could and got it down to just over hours. Parallelizing them would have been a lot of effort to set up the appropriate shared resources and configurations. Eventually, we set up multiple virtual machines so that the entire parallel test run could finish in about an hour and a half. But the total test time of all those runs sequentially is now almost ten hours and my working theory is that it's due to the overhead of the VMs on the host machine.
What I mean here is kind of like microservices applied to the various components of your application. You have an application that needs a database, a queue, a search components, and a cache. You could spin up a VM and install all those pieces. Or you could run a Postgres container, a RabbitMQ container, an ElasticSearch container, and a Redis container and leave your machine solely for the code.
When it comes right down to it, Docker containers are basically practical virtual machines. I've used VMs for many years. When I first started out, it was VMWare WorkStation on Windows. People that are smarter than me (including those that would notice that I should have said, "smarter than I") told me to use them. "One VM per client" they would say. To the point that their host was limited to checking email and Twitter clients.
I tried that and didn't like it. I didn't like waiting for the boot process on both the host and each client and I didn't like not taking full advantage of my host's hardware on the specific client I happened to be working on at that moment.
But containers are lightweight. Purposefully so. Delightfully so. As I speak, the overworked USB drive that houses my VMs is down to 20 GB of free space. I cringe at the idea of having to spin up another one. But the idea of a dozen containers I can pick and choose from, all under a GB? That's a development environment I can get behind.
Alas, this is mostly a future world I'm discussing. Docker is Linux only and I'm in the .NET space. So I have to wait until either: a) ASP.NET is ported over to Linux, or b) Docker supports Windows-based containers. And it's a big part of my excitement that BOTH of those conditions will likely be met within a year.
In the meantime, who's waiting? Earlier, I mentioned Postgres, Redis, ElasticSearch, and RabbitMQ. Those all work with Windows regardless of where they're actually running. Furthermore, Azure already has pre-built containers with all of these.
Much of this will be the basis of my talk at the upcoming MeasureUP conference next month. So...uhhh....don't read this until after that.
]]>In the meantime, I have been talking about them quite a bit recently. We've been using them on a project at Clear Measure and I'd like to think it's been successful but it's too soon to tell. I feel good about what we've done which, historically, has always been a good metric for me.
The topic has been covered at a technical and architectural level pretty well by Martin Fowler, so much so that he's even collected his discussions into a nice little Microservices Resource Guide. In it, he and other ThoughtWorkians define them (to the extent that anything in software containing the word "services" can be defined), point out pros and cons compared to monolithic applications, describe testing strategies, and cover off the major success stories in the space.
That doesn't leave much ground for me to cover which, from a marketing standpoint, is almost surely the point. But I would like to add my voice if for no other reason than to plug the podcast on the subject.
One of the more interesting links on Fowler's Resource Guide is tucked away at the bottom. It's a series of posts on how SoundCloud is migrating from a monolith to microservices. Part 1 discusses how they stopped working on the monolith and performed all new work in new microservices and part 2 is on how they split the monolith up into microservices. There were challenges in both cases, leading to other architectural decisions like event sourcing.
The arguments for and against are, predictably, passionate and academic. "Overkill!" you say. "Clean boundaries!" sez I. "But...DevOps!" you counter. "Yes...DevOps!" I respond. But SoundCloud's experience, to me, is the real selling point of microservices. Unlike Netflix and Amazon, it's a scale that is still relatable to many of us. We can picture ourselves in the offices there making the same decisions they went through and running up against the same problems. These guys have BEEN THERE, man! Not moving to microservices because they have to but because they had a real problem and needed a solution.
Now if you read the posts, there's a certain finality to them. "We ran into this problem so we solved it by doing X." What's missing from the narrative is doubt. When they ran into problems that required access to an internal API, did anyone ask if maybe they defined the boundaries incorrectly? Once event sourcing was introduced, was there a question of whether they were going too far down a rabbit hole?
That's not really the point of these posts, which is merely to relay the decision factors to see if it's similar enough to your situation to warrant an investigation into microservices. All the same, I think this aspect is important for something still in its relative infancy, because there are plenty of people waiting to tell you "I told you so" as soon as you hit your first snag. Knowing SoundCloud ran into the same doubt can be reassuring. Maybe I'm just waiting for Microservices: The Documentary.
Regardless, there are already plenty of counter-arguments (or more accurately, counter-assumptions) to anecdotal evidence. Maybe the situation isn't the same. They have infrastructure. They have money and time to rewrite. They have confident, "talented" developers who always know how to solve architectural problems the right away.
So now I've more or less done what I always do when I talk microservices, which is talk myself into a corner. Am I for 'em or agin 'em? And more importantly, should you, reader, use them?
The answer is: absolutely, of course, and yes. On your current project? That's a little murkier. The experience is there and microservices have been done successfully. It's still a bit of a wild west which can be exciting if you ain't much for book learnin'. But "exciting" isn't always the best reason to decide on an architecture if someone else is paying the bills. As with any architectural shift, you have to factor in the human variables in your particular project.
For my limited experience, I like them. They solve one set of problems nicely and introduce a new set of problems that are not only tractable, but fun, in this hillbilly's opinion.
And why else did you get into the industry if not to have fun?
]]>The side project is Western Devs, a fairly tight-knit community of developers of similar temperament but only vaguely similar backgrounds. It's a fun group to hang out with online and in person and at one point, someone thought "Wouldn't it be nice to build ourselves a website and have Kyle manage it while we lob increasingly ridiculous feature requests at him from afar?"
Alas, I suffer from an unfortunate condition I inherited from my grandfather on my mother's side called "Good Idea At The Time Syndrome" wherein one sees a community in need and charges in to make things right and damn the consequences on your social life because dammit, these people need help! The disease is common among condo association members and school bus drivers. Regardless, I liked the idea and we're currently trying to pull it off.
The first question: what do we build it in? WordPress was an option we came up with early so we could throw it away as fast as possible. Despite some dabbling, we're all more or less entrenched in .NET so an obvious choice was one of the numerous blog engines in that space. Personally, I'd consider Miniblog only because of its author.
Then someone suggested Jekyll hosted on GitHub pages due to its simplicity. This wasn't a word I usually assocated with hosting a blog, especially one in .NET, so I decided to give it a shot.
Cut to about a month later, and the stack consists of:
Of these, the one and only technology I had any experience with was Rake, which I used to automate UI tests at BookedIN. The rest, including Markdown, were foreign to me.
And Lord Tunderin' Jayzus I can not believe how quickly stuff came together. With GitHub Pages and Jekyll, infrastructure is all but non-existent. Octopress means no database, just file copying. Markdown, Slim and SASS have allowed me to scan and edit content files easier than with plain HTML and CSS. The Minimal Mistakes theme added so much built-in polish that I'm still finding new features in it today.
The most recent addition, and the one the prompted this post, was Travis. I'm a TeamCity guy and have been for years. I managed the TeamCity server for CodeBetter for many moons and on a recent project, had 6 agents running a suite of UI tests in parallel. So when I finally got fed up enough with our deploy process (one can type git pull origin source && rake site:publish
only so many times), TeamCity was the first hammer* I reached for.
One thing to note: I've been doing all my development so far on a MacBook. My TeamCity server is on Windows. I've done Rake and Ruby stuff on the CI server before without too much trouble but I still cringe inwardly whenever I have to set up builds involving technology where the readme says "Technically, it works on Windows". As it is, I have an older version of Ruby on the server that is still required for another project and on Windows, Jekyll requires Python but not the latest version, and I need to install a later version of DevKit, etc, etc, and so on and so forth.
A couple of hours later, I had a build created and running with no infrastructure errors. Except that it hung somewhere. No indication why in the build logs and at that moment, my 5-year-old said, "Dad, let's play hockey" which sounded less frustrating than having to set up a local Windows environment to debug this problem.
After a rousing game where I schooled the kid 34-0, I left him with his mother to deal with the tears and I sat down to tackle the CI build again. At this point, it occurred to me I could try something non-Windows-based. That's where Travis came in (on a suggestion from Dave Paquette who I also want to say is the one that suggested Jekyll but I might be wrong).
Fifteen minutes. That's how long it took to get my first (admittedly failing) build to run. It was frighteningly easy. I just had to hand over complete access to my GitHub repo, add a config file, and it virtually did the rest for me.
Twenty minutes later, I had my first passing build which only built the website. Less than an hour later and our dream of continuous deployment is done. No mucking with gems, no installing frameworks over RDP. I updated a grand total of four files: .travis.yml, _config.yml, Gemfile, and rakefile. And now, whenever someone checks into the source
branch, I am officially out of the loop. I had to do virtually nothing on the CI server itself, including setting up the Slack notifications.
This is a long-winded contradiction of my post of four years ago where my uncertainty with Java drove me to the comfort of .NET. And to keep perspective, this isn't exactly a mission critical, LOB application. All the same, for someone with 15-odd years of .NET experience under his obi, I'd be lying if I said I wasn't amazed at how quickly one can put together a functional website for multiple authors with non-Microsoft technology you barely have passing knowledge of.
To be clear, I'm fully aware of what people say about these things. I know Ruby is a fun language and I feel good about myself whenever I do anything substantial with it. And I know Markdown is all the rage with the kids these days. It's not really one technology on its own that made me approach epiphaniness. It's the way all the tools and libraries intermingle so well. Which has this optimistic hillbilly feeling like his personal life and professional life are starting to mirror each other.
Is there a lesson in here for others? I hope so as it would justify me typing all this out and clicking publish committing to the repository. But mostly, like everything else, I'm just happy to be here. As I've always said, if you've learned anything, that's your fault, not mine.
Kyle the Coalescent
* With credit to Brendan Enrick's and Steve Smith's Software Craftsmanship Calendar 2016
]]>I don't recall all the participants but it was a good number of the people on this list. Here, I'm going to attempt to summarize the salient points but given my memory, it'll more likely be a dissertation of my own thoughts. Which is just as well as I recall doing more talking than I should have.
This was a common thread throughout. Anyone who has done a significant amount of UI testing has asked a variant of this question. Usually in the form, "Why the &*%$ am I doing this?"
Let it not be said that UI testing is a "set it and forget it" affair. Computers are finicky things, UI's seemingly more so. Sometimes things can take just that one extra second to render and all of a sudden your test starts acting out a Woody Allen scene: Where's the button? There's supposed to be a button. YOU TOLD ME THERE WOULD BE A BUTTON!!!
Eventually, we more or less agreed that they are probably worth the pain. From my own experience, working on a small team with no QA department, they saved us on several occasions. Yes, there are the obvious cases where they catch a potential bug. But there was also a time when we had to re-write a large section of functionality with no change to the UI. I felt really good about having the tests then.
One counter-argument was whether you could just have a comprehensive suite of integration tests. But there's something to be said for having a test that:
This kind of integration test is hard to do, especially when you want to verify all the little UI things in between, like whether a success message showed up or whether the number of items in the shopping cart incremented by 1.
We also had the opposite debate: If you have a comprehensive suite of UI tests and are practicing BDD, do you still need TDD and unit tests? That was an interesting side discussion that warrants a separate post.
...is ongoing. There's no getting around that. No matter how bullet-proof you make your tests, the real world will always get in the way. Especially if you integrate with third-party services (<cough>PayPal<cough>). If you plan to introduce UI tests, know that your tests will be needy at times. They'll fail for reasons unknown for several consecutive runs, then mysteriously pass again. They'll fail only at certain times of the day, when Daylight Savings Time kicks in, or only on days when Taylor Swift is playing an outdoor venue in the western hemisphere. There will be no rhyme or reason to the failures and you will never, ever be able to reproduce them locally.
You'll add sleep
calls out of frustration and check in with only a vague hope that it will work. Your pull requests will be riddled with variations of "I swear I wouldn't normally do this" and "I HAVE NO IDEA WHAT'S GOING ON". You'll replace elegant CSS selectors with XPath so grotesque that Alan Turing will rise from his grave only to have his rotting eyeballs burst into flames at the sight of it.
This doesn't really jibe with the "probably worth it" statement earlier. It depends on how often you have to revisit them and how much effort goes into it. From my experience, early on the answer to that is often and a lot. As you learn the tricks, it dwindles significantly.
One of those tricks is the PageObject pattern. There was universal agreement that it is required when dealing with UI tests. I'll admit I hadn't heard of the pattern before the discussion but at the risk of sounding condescending, it sounds more like common sense than an actual pattern. It's something that, even if you don't implement it right away, you'll move toward naturally as you work with your UI tests.
...is hard, too. At least in the .NET world. Tools like Tarantino can help by creating scripts to prime and tear down a database. You can also create an endpoint (on a web app) that will clear and reset your database with known data.
The issue with these approaches is that the "known" data has to actually be known when you're writing your tests. If you change anything in it, Odin knows what ramifications that will have.
You can mitigate this a little depending on your technology. If you use SpecFlow, then you may have direct access to the code necessary to prime your database. Otherwise, maybe you can create a utility or API endpoints that allow you to populate your data in a more transparent manner. This is the sort of thing that a ReST endpoint can probably do pretty well.
Consensus for UI testing on mobile devices is that it sucks more than that time after the family dinner when our cousin, Toothless Maggie, cornered---...umm... it's pretty bad...
We would love to be proven wrong but to our collective knowledge, there are no decent ways to test a mobile UI in an automated fashion. From what I gather, it's no picnic doing it in a manual fashion. Emulators are laughably bad. And there are more than a few different types and versions of mobile device so you have to use these laughably bad options about a dozen different ways.
What about companies that will run through all your test scripts on multiple browsers and multiple devices? You could save some development pain that way. But I personally wouldn't feel comfortable unless the test scripts were extremely prescriptive. And if you're going to that length, you could argue that it's not a large effort to take those prescriptive steps and automate them.
That said, you might get some quick bang for your buck going this route. I've talked to a couple of them and they are always eager to help you. Some of them will even record their test sessions which I would consider a must-have if you decide to use a company for this.
I ain't gonna lie. I like Cucumber and Capybara. I've tried SpecFlow and it's probably as good as you can get in C#, which is decent enough. But it's hard to beat fill_in 'Email', :with => 'hill@billy.edu'
for conciseness and readability. That said, do not underestimate the effort it takes to introduce Ruby to a .NET shop. There is a certain discipline required to maintain your tests and if everyone is scared to dive into your rakefile, you're already mixing stripes with plaid.
We also discussed Canopy and there was a general appreciation for how it looks though Amir is the only one who has actually used it. Seems to balance the readability of Capybara with the "it's still .NET" aspect of companies that fear anything non-Microsoft.
Of course, there's Selenium both the IDE and the driver. We mentioned it mostly because you're supposed to.
Some version of Visual Studio also provided support for UI tests, both recorded and coded. The CodedUI tests are supposed to have a pretty nice fluent interface and we generally agreed that coded tests are the way to go instead of recorded ones (as if that were ever in doubt).
Ed. note: Shout out to Protractor as well. We didn’t discuss it directly but as Dave Paquette pointed out later, it helps avoid random Sleep calls in your tests because it knows how to wait until binding is done. Downside is that it’s specific to Angular.
Also: jasmine and PhantomJS got passing mentions, both favorable.
This is about as close as we got to disagreement. There was a claim that UI tests shouldn't be included in CI due to the length of time it takes to run them. Or if they are included, run them on a schedule (i.e. once a night) rather than on every "check in" (by which we mean, every feature).
To me, this is a question of money. If you have a single server and a single build agent, this is probably a valid argument. But if you want to get full value from your UI tests, get a second agent (preferably more) and run only the UI tests on it. If it's not interfering with your main build, it can run as often as you like. Yes, you may not get the feedback right away but you get it sooner than if you run the UI tests on a schedule.
The main takeaway we drew from the discussion, which you may have gleaned from this summary, is: damn, we should have recorded this. That's a mistake we hope to rectify for future discussions.
]]>