Book Review – Building Microservices

April 30th, 2015 by Ricky No comments »

Building Microservices Book CoverAt work, we’ve toyed with the idea of splitting our monolithic codebase into microservices. In my past life, I did a lot of SOA work using SOAP. In general the ideas interested me, but I moved on to grad school and working in the mobile space. Ultimately, I’ve been doing a lot of backend development; both at work but also to support API servers for mobile clients.

I picked up Building Microservices during an O’Reilly sale, not expecting much. This book has been quite the surprise. If you read through the O’Reilly reviews for this book, they do an accurate job of summing it up. This book does not go into specifics of programming language, or transport, or architecture really. However, it does a fantastic job with just the concepts. I know, a lot of times “the concepts” sounds like some high-level mumbo-jumbo. That said, this book cuts the right line of abstraction; presenting concepts just above the nitty gritty, extracting the maximum amount of content while leaving out the technology-specific portions.

For example, the book examines the differences between orchestration and choreography. To accomplish a business task, multiple services may need to work together to perform the workflow. The example from the book is registering a new user. In that scenario, the workflow included:

  • Creating a new user in the system
  • Sending a welcome packet to the user
  • Assigning loyalty points to the new user in the loyalty bank

All of these tasks are performed by separate services. The book presents two approaches for coordinating these actions: orchestration and choreography.

Orchestration is where a “master” process understands all the steps needed to complete the workflow, as well as the services that can perform these tasks. The master then orchestrates the execution of all the services. Choreography is a more reactive approach. With choreography, the customer service would emit a “customer created” event, perhaps to a message queue. Other services that are interested in such events can subscribe to them (perhaps via the queue) and act appropriately.

This discussion of orchestration vs. choreography is just one small section of the book. There are many more topics treated equally well throughout this book. If you’re interested in microservices, or things like Docker or containers I strongly recommend this book.

The Flight Diaries

October 21st, 2014 by Ricky No comments »
Mountaineer stadium, from the air

The four-to-six thousand foot view

For my birthday this year, Jodi got me perhaps one of the greatest presents: my introductory flight lesson. My first lesson was at Hart Field in Morgantown. The flight school at the airport is called RSA Flight Training, and it was amazing.

Getting my pilot’s license has always been one of my dreams, but I never looked into what it actually takes to get a license. My goal with this blog post, and the series to follow, is to detail the steps of my training. This should include every lesson and every dollar I sink into the training. So far I have about three hours of flight training and have spent about $600. I plan to create a page that gives a breakdown of all my spending and progress. Once that’s in place, I’ll drop the link here.

All updates will be tagged with “the flight diaries” tag in my WordPress system. That’s it for now.

So, you’re a developer? Welcome to devops.

November 17th, 2013 by Ricky No comments »

As far back as I can remember, I've enjoyed working with computers. Although I learned BASIC early on, I really got into computers with my friend Jimmy. He taught me to tear into them, rebuild and upgrade them. He and I were both introduced to Linux in high school. However, when I started writing software I moved away from hardware and system administration. I've recently learned I'll be doing both simultaneously, probably for the rest of my career.

For a lot of my career, system administration was a separate task from software development. Systems were either there only to support our software (for example, for demo purposes), or were separately maintained. Sure, I could maintain a Windows box, or I could configure a DHCP server on Linux, but these tasks were either handled by someone else or unimportant to the task at hand.

However, while building our platform at Shake I was responsible for both system administration and software development. While building the Shake backend I was very drawn to continuous deployment infrastructures. Things like Heroku make deployment simple and system-administration a non-issue. This worked great, for a while.

Unfortunately, we ended up running into issues with the host we deployed on (and no, it wasn't Heroku). Although we had shell access to our provider, when you abstract away your platform as part of a deployment model you lose the flexibility of being able to administer your boxes. In addition, we were also abstracted from the realization that our box was shared with many (possibly thousands) of other web applications.

When building a production web application, the idea of migrating the application to another location is painful. I'm willing to go out on a limb and say migration is to be avoided if at all possible. However, if that migration means you have more flexibility and control over downtime, then it becomes much more appealing. It doesn't make the process any easier, though.

Which brings us to now. I really missed the train on webapps and cloud-deployment. Today's developers need to have a command of both software development AND a level of comfort with system administration. While I think you could survive with a somewhat barebones amount of sysadmin knowledge, the stronger you are in that area the better a developer you are overall.

So, if you love software development, it's time to get comfortable with system administration. Building you're own computer is fun, but no longer necessary. Since Linux is basically the operating system of the web, it's time to get comfortable with sysadmin tasks (and creating a Linux machine). Below is a list of a few books that may help you along your way. Some of the books I have personally used (and may be a bit outdated), while others have been recommended via the web. At the least, check them out.

Unix in a Nutshell, Fourth Edition
I used this book many years ago. It goes into detail about different commands and shells. It's got some good info in it, but may not be the best introduction.

UNIX and Linux System Administration Handbook (4th Edition)
I have not read this book, but it appears to be well received on the web.

Essential System Administration: Tools and Techniques for Linux and Unix Administration, 3rd Edition
I haven't read this one, either, but again, it appears to have favorable reviews online.

Why You Shouldn’t Waste Your Time

January 8th, 2012 by Ricky No comments »

Just a quick re-affirmation and follow-up to the previous post. On the way home from my cousin’s birthday party, I was listening to an episode of This American Life titled Ruining it for the Rest of Us. In the prologue, they talk about a research demonstrating that attitudes of team members can totally poison a team. Even a team of really good people.

Essentially, the research identifies three personalities: jerk, slacker and depressive. By introducing an actor exhibiting these traits into an otherwise functional team, the team productivity was reduced by as much as 40%. Just one person, that’s all it took.

The interesting thing is, the team was totally poisoned by attitude. It wasn’t incompetence, it was attitude. So, let’s say your in this position, and your exhibiting these behaviors, being a jerk, or being depressed, or slacking, because of the work you do, it makes it that much harder to do a good job. You have to spend effort on changing your attitude, and not doing work. It’s a division of interest.

I guess what I’m saying is, find what you love to do, and make it work. You won’t have to spend effort on changing your attitude. An important part of that is finding a team to work with which feels the same way. I believe, at that point, the work will come easy, and you’ll remove another barrier to success.

Starting Over

January 8th, 2012 by Ricky 3 comments »

So I’m starting over. Again.

Last week, I let my PhD advisor know I wouldn’t be returning to school this semester. I spent a lot of time thinking this decision over during the holidays. I agonized. I talked to a lot of trusted friends. I talked to some of my family. I let them know I was considering dropping out of the PhD program, and asked what they thought. Everyone was supportive, and the most common response, in some form or another, was, “you never really seemed excited about your research in the first place.”

So, here I am again. It feels strangely familiar, like 2008 all over again. For context, in 2008 I left a relatively comfortable government contracting job to return to academia. I liked the people, I just wasn’t crazy about the work. After that experience, I told myself when I started doing work that I didn’t enjoy, I’d find something else. Well, this weekend I decided it was time to find something else.

I’ve been doing a little bit of independent iOS-related contracting throughout the past semester. Since working at foursquare this summer, I solidified that I really enjoy mobile development. Mobile is still an emerging space. Right now, there are plenty of chances to help define what the mobile world looks like. There are still plenty of problems to solve, and I’d like to be a part of that.

I believe the mobile space will be relatively solidified in four years. Hell, we could see it shake-out in the next two years. If I don’t make a move now, I feel I’ll be missing out on an opportunity to help define an industry. On the other hand, academia will be there if and when I decide to come back. I’ve left the industry before, and if it came down to it, I could do it again.

When I’m at the end of my life, I’d like to look back at and see a portion where I was a teacher. I still want to teach. But at the same time, I don’t want to look back and regret not taking a chance to help shape the future, in whatever magnitude I can. So, now, I’m off into the unknown once again. I think I can make ends meet for the time being. I just don’t know how to legally pay myself from my business account. Any ideas?

foursquare – week #1

May 23rd, 2011 by Ricky 6 comments »

So, I’m in New York City. Unless you know me, you probably wouldn’t know that. And I guess if you didn’t know me, then maybe this wouldn’t be surprising information. But I’m not from New York City, I’m from West Virginia. What am I doing here? I’m grabbing an awesome opportunity with a death-grip and running with it. I’m an intern at foursquare.

Back in March I applied to the NYC Turing Fellowship and somehow, despite an enormous number of applicants, managed to be accepted to the semi-finals. During this round applicants spent a weekend in New York interviewing with a host of incredible startups. I was fortunate enough to interview with some amazing companies, but coming into the weekend I had my heart set on foursquare. After returning from the weekend in New York, it was an anxious several days. Like after a first date, when your unsure of when to call the other person, or wrestling with the possibility they were totally uninterested in you. Fortunately, after what seemed to be a lot longer than what it actually was, I received a response from the NYC Turing Fellowship: foursquare had agreed to let me come on as an intern for the summer.

I won’t get into the details of the logistics of coming to New York for the summer, or what my first impressions of the city were, or anything like that. That’s grist for another mill, or at least another blog post. But what I will lay out is why I wanted to work for foursquare, and what my first week was like.

foursquare is an interesting concept. If you’re not familiar, it’s a location-based game that keeps track of where you’ve been, and where your friends have been, awarding points for exploring new places and keeping a leaderboard of the users, as well as providing how you rank against your friends. There are other aspects like mayorship and specials to help complete the compulsion loop, but I’ll just let you read the foursquare about page.

What I was most intrigued by was the concept of location as part of our identities. After a decade and a half online, people my age are starting to understand the value of having an online presence. What do I mean? I mean, having an email address, maintaining a website, using some form of instant communication like AIM or Skype. I feel that folks around my age, who grew up with the explosion of the internet, originally started out carefully defending the information they posted online. Now, the benefit of having an accessible presence online, and instantly being able to access information from others in your network, is invaluable. This is social networking, and my peers are finally starting to understand the value of it. Folks younger than I generally seem to have skipped the fear of posting information about yourself and caught right on.

foursquare is unique in that it understands that location is an important part of our identity. Just as we share our email address to receive the benefit of connected communication, foursquare understands physical location is a very important part of our identity, and allowing us to share that information with our friends is invaluable. It also helps that mobile is a big part of their strategy. I love mobile.

So, what was the first week like? I’d be dishonest if I told you I walked in totally confident. However, upon arriving in the office I was greeted by Susan, who I’d interviewed with, and was given a quick tour of the office. Employees sit in rows of desks, next to each other. There’s a fridge and mini-kitchen. There are Sonos players for music. It’s hard to describe but the environment is very energetic. I also snagged a fousquare hoodie and tshirt, so I can’t complain about that.

I spent the next half hour being introduced and meeting the iOS team, whom I’d be working with over the next couple months. I was refreshed that, throughout the day, I was approached cordially by almost everyone in the office. They introduced themselves, asked what I thought I’d be working on, and in most cases mentioned they were excited to have me onboard. How cool is that? Literally, everyone in the office, from my team to the designers to  the business folks, even Dennis and Naveen, the founders.

The first day I was little awkward. I wasn’t user what my tasking would be. After setting up my machine I peered with Anoop, the iOS team lead, to fix a bug and explore the code. I typically like to start working with a new code base by fixing a few bugs, if possible. It helps you find the execution flow through the code, but typically gives a more macro level understanding of the product: was this a user-facing issues?  how was it discovered? what functionality does it affect? I spent the next day or so working on a few bugs.

Unfortunately, I think I wasn’t quite yet in the right mindset. Internally, at least, I was waiting for some tasking. Some list of things to do. What I didn’t understand is that I was now onboard with a startup. It’s time to contribute. While there is direction and vision, as far as tasking and goals go I’m allowed to design my own future. I get to bring my ideas to the table, and sell them to the team. Somewhere, a form of me was looking for a checklist of to-do’s, when really I was given a blank canvas. It took a few days to realize, but now I think I finally understand.

So that was my first week at foursquare. There was a bit of flailing on my part, taking me a while to realize the sort of responsibility I’ve been given, but I’m super-excited. After talking to the engineers and other folks there, and researching some of the resources available I have a very high-level idea of what I’d like to do. I’m invested. I think this product has the potential to be incredible, to exploit some extra value in foursquare, but it’s up to me to execute and iterate on that idea.

After this first week and this incredible opportunity, I’ve had some time to reflect on the entire process. A lot of friends and developers back home reminded me that I’m very fortunate, and what an incredible opportunity this is. It’s true. But I wouldn’t have this chance without the existence of the NYC Turing Fellowship. I’m humbled they saw something in me that qualified me for the position, and the other students I met while interviewing were all sharp, outgoing kids from some spectacular schools. It’s an honor to be part of the program, and a privilege to be spending my summer at an awesome place like foursquare.

Lovely Ride – Redux

December 27th, 2010 by Ricky No comments »

Introduction
For those few that read the sporadic postings on this blog, you may have read a previous article back in July when I decided to start a business. I won’t say the decision was hasty; I was reasonably financially secure, and planned to hedge my bets using student loan funding. Since I’ve finished my coursework for my masters, I imagined that taking an extra year to finish the program would grant ample time to devote to my new business. In the original post I had planned to give myself runway until December. Well, it’s December. This post is a redux of my experience over the past five months.

To speak briefly about my motives, I wanted to start a business around mobile software for a variety of reasons. First, the market is experiencing rapid growth. Much like the first generation of the internet, it is important to establish identity and presence online. This same situation is now reoccurring in the mobile space. Second, I wanted to stretch my legs technically in mobile application and web development. Third, I had recently rediscovered my love of software development. Having my own business meant that I could devote as much time as I wanted to my own projects, whether they were contract work or developing applications for my brand. I believed, and still do, that I don’t need to compromise my values to make a living developing software.

Contract Work

I didn’t start the process entirely on a whim. I had one potential contract in hand to help get my business off the ground. Within a couple weeks, I had two potential contracts on radar. While I wanted to focus on developing my software and brand, I was not about to turn down business. Unfortunately, as the old aphorism says, one in the hand is worth two in the bush. As it would turn out in this case, two in the bush are worth zero in the hand. Both leads ended up fizzling out, and unfortunately disappointing way. After continuous back-and-forth via email and almost instantaneous responses on my part, both leads just ended communication. I had even signed an NDA with one. Needless to say I was disappointed.

Application / Brand Development

In my back pocket I had an application that I was working on with a colleague. I pushed forward quite a bit on this application. I was excited about the technology and the niche the software could fill, albeit for a small market. In a similar fashion, though, communication with the other party was sparse and slow. After pinging my partner I would typically get a response, but little to no advancement on the project. To be totally truthful, I don’t think my partner is sandbagging, he just has a lot of other responsibilities on his plate. From my perspective on this particular project, though, there hasn’t been much help kicking the ball down the road. Normally I would continue on my own, but this project isn’t  my original idea and some previous verbal agreements were made. I can’t, in good conscience, adopt this project as my own. So, the sum result is a lot of time invested in what could be a stillborn.

In the Present

In the context of LovelyRide, that pretty much brings us to now. It’s now December, and due to a combination of a bit of liberal budgeting and not getting quite as much net out of my student loans I am barely scraping by financially. I’ll be able to tide over until my next loan disbursement, but the tail end of this month has been much tighter than I would have liked. I have not been able to launch the application mentioned in the previous paragraph. I have a lead on some work, and look forward to developing a relationship with this potential client and providing stellar software if we can agree in negotiations. Not the picture of success I had hoped for.

Lessons Learned and The Road Ahead

So, what have I learned? I think the biggest lesson I’ve learned is to always be on the hunt for new business. A verbal agreement to a lead, or even to work, is never set in stone. If pursuing contract work, don’t relent on the business development until something concrete is in place. In general, I should have managed my financial risk more actively. This is common sense, and I fell into the trap of wanting to jump to development without having the financial strings in place. In order to run a business, you need to RUN the business, not just sit behind a screen and type code. Also, I should have been more explicit in my agreement with my partner, or at least been aware of the risks of taking on such a project.

And what does the future hold? In the July post I mentioned that December was the end of my runway. Well, I’ve changed my mind. Now, May, the end of the spring semester will be the end of my runway. I receive another disbursement of loans in the spring which should pad me financially. As I mentioned above, I’ve received a ping from a potential client about a project. Most importantly, I have the opportunity to continue without much risk. Continuing for another six months won’t destroy me financially, and I still believe it’s feasible. While my success hasn’t quite been as I pictured it, that doesn’t mean I won’t envision wild success, I’m just going to be more intelligent and experienced about pursuing that vision. Here’s to another five months of learning.

Soft Landings

August 7th, 2010 by Ricky No comments »

I’m currently working on an iPhone project that requires a Mac helper application. The Mac application doesn’t require much user interaction. In fact, if a user is spending time interacting with the Mac application OR the iPhone application, something is wrong; the workflow needs to be extremely streamlined. Even with a streamlined interface, there’s one or two preferences the user can tweak. Since the user doesn’t regularly interact with the interface of the program, it would be awesome to show the user around the first time they launch the application.

This concept is called a soft landing. Soft landings provide a user hints or assistance the first time they launch a program. What soft landings specifically provide will vary depending on the application and workflows. For instance Memory Miner, an application for linking media together from the iLife suite, provides a sample library of photos. This makes good sense. Memory Miner focuses on media. Asking the user to create their own media library to even BEGIN using the application is a bad, bad idea.

Video games are the best examples of soft landings. If you’re old enough to remember playing Atari or NES games you’d know that reading the instruction manual before playing a game was nearly a requirement. There were no soft landings; you insert the game cartridge and the game starts. There’s no help, you’re on your own. In today’s games this is blasphemy. I can’t think of a single game lacking some sort of tutorial or introductory level. Earlier this week I was playing Gears of War 2 and was struck by the elegance of the soft landing. Upon starting a new game you’re given the option to “train the rookie.” It’s basically a tutorial for the controls, but regardless of your choice the storyline progresses smoothly. Games are a great place to look for inspirational soft landings.

After three paragraphs of rambling, let’s talk about my app (this is MY blog, after all). As I mentioned before, my specific goal is to help the user interact with the application. First, a little background. My application will live in the menubar (for Windows folks, think system tray). I’d like the process to run while the user is logged in, like a daemon. It’s common for Mac application that run in this manner to do two things:

  1. Remove the app icon from the dock
  2. Launch when the user logs in

Choosing not to have a dock icon means the user will interact ONLY through the menubar icon. This can be a little confusing. Imagine launching an application and not seeing a displayed window, nor a dock icon to confirm it’s running. You’d probably assume it crashed and delete it. Priority number one is showing the user the app actually is running and that you interact with it via the displayed menubar icon. I need a splash screen.

Before the splash screen reveal, I need to make clear that I’m not a graphic designer. I realize that. I’m trying to focus on functionality here, not necessarily how “pretty” the screen looks. There were two points I wanted to emphasize. First, the location of the menubar icon that is used to interact with the application. Secondly I wanted to let the user know that the splash screen will only be shown once and can be closed at any time. I spent half a day sketching out screens on paper, taking notes on what worked and what didn’t between revisions. After arriving at a final sketch I moved to Pixelmator to design the screen you see below.

SongFlow's soft landing splash screen.

Click for full-size image

I’m not entirely pleased with it, I think there’s a lot of room for improvement. But, I believe it’s functionally accurate. The logo is in the center identifying the application. The eye is drawn to the top, suggesting the user look to the menubar and providing familiar icons to provide context. Lastly, the small text at the bottom explains that the window will only appear once.

I learned quite a bit from this exercise. First, sketching ideas on paper is extremely helpful. It allowed me to ignore color schemes and matching, which I am notoriously bad at, and concentrate on text and graphic placement. Also, I didn’t spend time worrying about how to use the Pixelmator tools to implement an effect, I could just scratch it down. Revising the design in my notebook allowed me to go through several iterations and collect my thoughts in just a couple hours; much faster than I can move in Pixelmator.

I also gained a respect for ratios and aspect. After my second revision I wanted to shoot for an iPhone metaphor, keeping the splash screen the same aspect ratio as the iPhone. I sketched it all out on paper in landscape in my notebook and was very pleased with the result. However, after moving to Pixelmator I quickly realized that ratio wouldn’t work on my monitor with the sizes I had in mind for the graphics and text. The window would simply be too long. Relaxing the metaphor and allowing the window to be a lot shorter and slightly wider produced the screen above.

Some of my designer friends have mentioned the importance of typography. I’ve always thought that a rather silly concept. That was until I started on this project. I spent quite a bit of time trying to find a font that looked professional yet casual enough to fit the idea of our application. I totally gave up on trying to mix different typographies. Much like color matching, I just don’t have the experience in that area.

In conclusion, this was a great exercise. It’s shown me the value of working on paper, as well as the value of developing experience with design, colors and typography. I’ve learned that if I can afford to shop work out to a designer, it’s probably worth it, but I can at least do paper sketching to provide a solid idea for the direction I’d like to move forward. I better understand where my strength and responsibilities lie, and where to hand the reigns off to someone else.

If you have any comments regarding the design, or other thoughts on how to guide the user I’d love to hear them. Also, if you’d like to take a stab at redesigning this screen (for your own enjoyment, of course) I’d be happy to provide the artwork.

Fear of the Unknown

July 19th, 2010 by Ricky 2 comments »

This weekend I filed the paperwork to create my first business. The state of West Virginia, while lacking in many areas, actually has quite a nice system for registering a business online. The process was entirely electronic, with one exception; I had to print off a signature page and fax or mail it to the secretary of state’s office.

This wasn’t entirely painful. I don’t own a fax, and I really don’t know anyone  that does. After printing and signing the documentation I stopped by Office Depot to fax the forms to Charleston. The fax cost me seven bucks. SEVEN BUCKS! This alone is now motivation to look into online fax services.

I completed online portion of the application at Starbucks over the course of two days; a half hour or so on Saturday and a couple hours on Sunday. Including printing and faxing the process couldn’t have take more than three hours. This afternoon I was alerted my application had been processed. Nice turnaround, Natalie Tennant (WV’s secretary of state).

I must admit I was nervous while filling out the application. Nervous I was going to fill out the paperwork improperly. As a result the IRS would burst down my doors, raid my apartment and jail me for fraud or tax evasion. It’s silly, I know. I’ve realized I tend to fear or worry over the unknown. Maybe this is typical of anyone.

Right now, for the first time in several years, I’m at a point of uncertainty in my life. I can’t really afford to stay in school any longer. I don’t have a job lined up for August, but I’m relatively certain I could find employment. It’s just the jobs I could find, while the pay would be reasonable, I’m not passionate about. Part of my returning to school for my masters was to move into a field I was excited about.

This business is a part of my “master plan.” I applied for financial aid, and have the opportunity to take out student loans for the fall. For those that don’t know, student loans are provided by the government at typically much lower interest than commercially available loans. I’m planning to pull out enough loan money to allow me to live for the next six months. My current plan is to launch my business and be profitable within six months. Actually, to be self-sufficient within six months.

As I was driving home this evening I was once again worrying about my plan. My worries included, but were not limited to; not having a website up yet, not having a product, not having any clients and going broke. These are all concerns. And they’re all valid concerns. But worrying isn’t going to accomplish much.

At this point I can do one of two things. Well, three things. I can worry, or I can push forward and try to solve problems (I could also work on things and worry about them, which is what will most likely happen). Worrying alone doesn’t do much except breed more worrying, fear and uncertainty.

So, for the time being I’m going to push forward. I may back out at the last minute. I may be cautious and take a safe 9-5 type job that pays well, but I hope not. I’ve done that before. I’d like to see if I can make this stuff happen on my own. I’d like to be self-made, or at least try. This worrying about the unknown crap just isn’t gonna cut it.

Also, thanks to my friend Adam over at Wordplay Software for helping me with some questions I had while filling out paperwork.

Ring Benchmark Implementation (pt. 2): Objective-C and Results

February 22nd, 2010 by Ricky 5 comments »

This time I’ll present the Objective-C implementation of the ring benchmark. This implementation relies heavily on Grand Central Dispatch, or GCD. As part of the GCD implementation, blocks will be used as the “units of work,” both for the processes in the ring as well as the “setup” code.

First, some support code. I stole the CHECK macro below from Mike Ash. Mike has an excellent blog, full of great Mac programming info, and I suggest you check it out.

// The CHECK macro and function were stolen from the intelligent
// Mr. Mike Ash
#define CHECK(call) Check(call, __FILE__, __LINE__, #call)

static int Check(int retval, const char *file, int line, const char *name)
{
    if(retval == -1)
    {
        fprintf(stderr, "%s:%d: %s returned error %d (%s)\n", file, line, name, errno, strerror(errno));
        exit(1);
    }
    return retval;
}

Let’s start with the main() function:

int main (int argc, char * const argv[]) {
    __block NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
	__block uint64_t start;

	// The number of processes in the ring
	int numProcs = 0;
	
	// The number of transitions the message should make
	int sendCounter = 0;
	
	// Parse the command line options
	parse_options(argv, argc, &sendCounter, &numProcs);
	
	// Use socketpair to generate a pair of connected sockets.
	// Generate a pair for the communication between each
	// process in the ring and it's neighbors.
	int socketArray[(numProcs+1)*2];
	
	for (int i=0; i<(numProcs+1); i++) {
		CHECK(socketpair(PF_LOCAL, SOCK_STREAM, 0, socketArray+2*i));
		debug_NSLog(@"Created socketpair %d and %d", socketArray[i*2], socketArray[i*2+1]);
	}

	debug_NSLog(@"Debugging!");
	NSLog(@"Initializing a %d thread ring to send a message around %d times", numProcs, sendCounter);
	start = mach_absolute_time();
	
	// Later, the sources will need to be canceled, so
	// the NSArray is used to keep track of them
	NSMutableArray *dispatchSources = [[NSMutableArray alloc] init];	
	
	setup_ring_dispatch_sources(dispatchSources, numProcs, socketArray);
	setup_main_dispatch_source(dispatchSources, numProcs, socketArray, start);
	
	int sendto = 2;
	
	// Send the initial message to the process ring. Once
	// the message is sent the GCD threads will 'do their thing'
	// and receive the data
	CHECK(write(socketArray[sendto], &sendCounter, sizeof(int)));

	// This starts the main dispatch queue and essentially
	// puts the train in motion
	dispatch_main();

	// ... and never make it to this point
    return 0;
}

I would encourage you to click the "view source" icon (looks like a "less than" and "greater than" arrow when you hover over the source snippets). This will open the source in another window, and makes it a bit easier to follow along with the rest of this post.

The first bit of code just parses off options specified on the command line, specifically the number of processes and the number of times the message should be passed around the loop (not included in this snippet of code).

The first interesting portion of code is the socketpair() call. Sockets will be the mechanism used for interprocess communication (or IPC from here on out). socketpair() will provide two connected socket file descriptors into the int array passed in. I used some pointer arithmetic inside the for loop to generate several sets of connected sockets.

The loop will generate numProcs + 1 connected sockets. If there are numProcs processes participating, then to connect them in a loop we need numProc sets of sockets (numProcs - 1 to connect them in a line, and one more to connect the first and last process in a loop). The additional socket pair is created to allow the main thread to initiate the message passing, yielding a total of numProcs+1 pairs of sockets.

Next, the timer is captured and GCD is setup to run the processes in the ring as well as the main loop. The NSMutableArray is created to keep track of these processes, as they need to be properly "shut down" when the program is finished. The last two bits of the main function start the message passing and then fall off onto the main thread with the dispatch_main() call. As mentioned in the comments, dispatch_main() never returns; after this call all execution will happen through the GCD queues.

So, with a feeling for the main execution, let's look at setting up the GCD processes:

// Sets up the ring dispatch sources
void setup_ring_dispatch_sources(NSMutableArray* sources, int numProcs, int * socketArray) {

	// This code generates the dispatch_sources
	// Dispatch sources will wait to read from a file
	// descriptor. In this case, a socket.
		
	for (int i=3; i<2*(numProcs+1); i+=2) {
		dispatch_queue_t globalQueue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
		dispatch_source_t socketSource = dispatch_source_create(DISPATCH_SOURCE_TYPE_READ, socketArray[i], 0, globalQueue);
		
		// This block will be executed when data is available for
		// reading on the socket file descriptor.
		dispatch_source_set_event_handler(socketSource, ^{
			
			int rcvCounter = 0;
			int rcvLen = read(socketArray[i], (void*)&rcvCounter, sizeof(int));
			
			if (rcvLen > 0) {
				debug_NSLog(@"Got int:%d from socket %d", rcvCounter, socketArray[i]);
				
				// If the rcvCounter = 0, let the main thread know the
				// process is complete
				if (!rcvCounter) {
					CHECK(write(socketArray[0], &rcvCounter, sizeof(int)));
					
					// Setup detail. Connect the end of the ring to the
					// beginning. Hence, 'ring'
				} else if (i == ((numProcs+1)*2)-1) {
					--rcvCounter;
					CHECK(write(socketArray[2], &rcvCounter, sizeof(int)));
					
					//  This is the normal case in the ring. Send the
					// message to the next 'process' in the ring
				} else {
					--rcvCounter;
					CHECK(write(socketArray[i+1], &rcvCounter, sizeof(int)));
				}
			}
		});
		
		// As mentioned above, add the source to the array and
		// kick it off
		[sources addObject:[NSValue valueWithPointer:socketSource]];
		dispatch_resume(socketSource);
	}	
}

setup_ring_dispatch_sources()'s responsibility is to setup the GCD dispatch queues for the process ring. There's a bit of vocabulary to catch up on to understand this section. A dispatch source is a construct that is attached to some sort of file descriptor. When the file descriptor enters a certain state, the dispatch source will execute some code. Specifically, with regards to this example, the file descriptor is the incoming socket for each process, and the code that will be executed will be to pass the message on to the next process. Also, GCD becomes much more elegant when used with blocks. Blocks are simply anonymous functions that allow for nice scoping properties (it's possible to inherit variables from a block's parent scope).

The dispatch_source_set_event_handler() is the function where a block is used, and contains the code that will be executed when data is available on the incoming socket. Much like the Erlang code from my previous post, there are three situations that should cover all processes in the ring:

  • The message is at the tail of the loop, and the traversal counter has reached zero. Message the main thread that execution is complete
  • The message is at the tail of the loop, but the traversal counter has reached zero. Decrement the counter and send the message back through the loop.
  • The message is somewhere in the loop, pass it on

Finally, the function keeps track of the dispatch_source_t structure for releasing once the program has finished executing, and the GCD dispatch queue is resumed. As Mike Ash pointed out, DISPATCH SOURCES MUST BE RESUMED AFTER CREATION. Apparently, when created they are not in a "resumed" mode. This can lead to frustration, so if you decide to include GCD in your next project keep that caveat in mind.

The last real bit of code is setting up the main thread's dispatch_source:

// Creates the 'father' thread that will initiate message passing
// and that the process ring will report to upon completion
void setup_main_dispatch_source(NSMutableArray* source, int numProcs, int * socketArray, uint64_t startTime) {
	
	// Create the main dispatch queue, which will run on
	/// the main thread...
	dispatch_queue_t mainQueue = dispatch_get_main_queue();
	
	// Create a dispatch source for the socket used
	// to communicate with the 'main' calling thread
	dispatch_source_t mainSocketSource = dispatch_source_create(DISPATCH_SOURCE_TYPE_READ, socketArray[1], 0, mainQueue);
	
	// Similar to the Erlang program, this code waits
	// for a message to come in on the main thread's socket.
	// When it does, the total runtime is calculated and
	// displayed
	dispatch_source_set_event_handler(mainSocketSource, ^{
		
		// This timebase info crap was a PAIN IN THE ASS
		// to find. I imagined it would be much easier to
		// find a high-precision timer. I was wrong.
		uint64_t end = mach_absolute_time();
		uint64_t elapsed = end - startTime;
		static mach_timebase_info_data_t sTimebaseInfo;
		
		if ( sTimebaseInfo.denom == 0 ) {
			(void) mach_timebase_info(&sTimebaseInfo);
		}
		
		uint64_t nanoSeconds = elapsed * sTimebaseInfo.numer / sTimebaseInfo.denom;
		NSLog(@"Finished! %.2f uS", (float)nanoSeconds/(float)1000);
		
		for (int i=0; i<2*(numProcs+1); i++) {
			CHECK(close(socketArray[i]));
		}
		
		// Cancel all dispatch sources
		for (NSValue *sourceAsValue in source) {
			dispatch_source_t source = (dispatch_source_t)sourceAsValue;
			dispatch_source_cancel(source);
		}
		
		// TODO: need to figure out what causes [pool drain] to crash		
		//		[pool drain];
		exit(EXIT_SUCCESS);
	});
	
	dispatch_resume(mainSocketSource);
}

This function is fairly straightforward. A block is attached to a dispatch_source on the main process's socket. This code will be executed when the message has been passed around the loop the specified number of times. The code gets the current time and calculates the program run-time.

And that's it for the Objective-C implementation. Before I get to the results I'd like to make a few observations. First, I think it's easily seen that the Erlang code is much more concise. With Objective-C we have to jump through the headaches of setting up sockets and using GCD and blocks. Easily the most confusing part of the Objective-C code is the strange socket array mangling and pointer arithmetic.

I did lose a little sleep on cleaning that portion of the code up, but ultimately decided against it, in the spirit of the challenge. I had hoped to expose more of the process of creating the two programs, not so much the final product. Thus, the kludginess of the socket file descriptors lends some insight to the development of these programs.

But, let's be fair here. I think this challenge was a bit loaded to showcase Erlang's strengths. Concurrency is built into the language. IPC is built into the language. This exercise is a fancy showcase for these two features.

Alright, that's enough of prattling on. Let's take a look at the results:

procs=10, loops=10 procs=100, loops=100 procs=1000, loops=100
Erlang ~100 uS ~1000 uS ~5000 uS
Objective-C ~5000 uS ~270 mS ~2.6 S



To compare these numbers, let's look at how much faster Erlang is on each of these trials:

Trial Parameters Erlang speed improvement
procs=10, loops=10 50x
procs=100, loops=100 270x
procs=1000, loops=100 520x


It's obvious that Erlang was built for concurrency. Even with the "state of the art" concurrency GCD provides for Objective-C, Erlang takes it to task. I should do a bit more investigation, looking into how Erlang scales with processes versus number of messages passed. I have a feeling Erlang scales relatively linearly with processes while messages have a negligible impact. At any rate, it was a fun exercise.

Here's the code