I Need an App

As a startup coach and as a freelance app developer, that’s a sentence I hear a lot. Entrepreneurs with a business idea, but not necessarily the tech know-how to make it a reality, want to know what it takes to develop a mobile application. And of course, like many things, it’s not as easy as it looks, there are many blind spots, a lot of misconceptions and depending on who you ask questions, you might get very different answers.

So in this post, I will try to cover all of the questions that I generally ask back to entrepreneurs who come with this statement. I will do it from my point of view and based on my experience as both a Lean Startup coach and a Belgian freelance software engineer. I co-founded the NEST’up startup accelerator a few years ago, I coached at least a dozen entrepreneurs, especially to Lean Startup principles and techniques, and as a developer I have developed a couple of apps. A few years ago, as a VP of Engineering for Take Eat Easy, I even supervised the development of a whole line-up of mobile apps for consumers, restaurants and couriers.

So if you want me to answer the big question, here are the “small” ones you need to think about first.

TLDR; This article is long, so if you want to skip to a shorter version of its content, here is a small document I prepared for you.

Do you really need an app?

Mobile apps are everywhere, we use them everyday, and as an entrepreneur, it’s easy to think they are necessary to any serious business model. But remember that a startup IS NOT an app, it CAN HAVE an app, that is only a part of your overall business model. It can be one of your sales channels, one of your promotion channels, one of your customer support channels, sometimes even part of your logistics, but you have many parts to figure out around all of that, many hypotheses to validate about your pricing model, your customer target, your marketing, your competition, etc. And if you count on a fully developed application to figure out all those things, you are taking a huge risk, because (sorry for the first revelation) AN APP IS FREAKING EXPENSIVE! And by that I mean developing a fully custom application, with your brand on it, and all of the features you dream about.

So the first question you should ask yourself is whether there aren’t any other creative ways to de-risk your investment, to validate your hypotheses about who your customers are, what problems you want to solve for them and what a solution to those problems could look like, without developing an app. There are dozens of examples of that in entrepreneurship history, from Zappos to Airbnb all the way through AngelList: a website or an app is not the most efficient way to figure out your business model early on. It can be part of a strategy, going through different forms of your products, getting progressively more and more sophisticated as you understand your market better and better, but it is rarely the first phase of that strategy. So think about mailing lists, setting up a simple phone number to respond to your customers’ requests directly, off-the-shelf Drupals or WordPresses or forums, a Slack or a Discord, a Google Form, or even a no-code app that you can configure yourself (Bubble, Quixy, Airtable, Webflow, Glide, etc.).

Granted, those early channels are often not very scalable (they are often called “concierge mode”), but remember that when you start up, your most precious capital is not “a lot of customers” or even money, but validated learning. It is all the things you learn about your market. Those validated hypotheses can get you funding (through sales and investors), they can prevent you from wasting valuable money and energy on developing features no one will use, they will be a competitive advantage over all the competitors who will just jump on making an app, and they will help you clarify and prioritize what your future app will really need, which will inform the initial UX design, which is a prerequisite to any budget discussion.

Of course, mobile app development agencies will rarely tell you that, but as an app developer, I would rather have you come back to me in a few months with a clearer idea of what your market needs in your app, and possibly even some funding, than trying to develop an app now based on a very vague idea of its design, no funding and a huge risk that I will end up developing a throw-away application and building a very stressful relationship with you.

Are you ready for a long-term commitment?

A lot of business-only entrepreneurs imagine developing an app or even a website as a one-time thing. They are looking for a magic budget number that they can add to the one-shot column of their business plan and call it a day. And unfortunately, some development agencies, who prefer working on fixed price projects, will go along with that misconception.

But the fact of the matter is that your app, just like your business model are living things, especially for a startup. Your market will change a lot around you, independently from anything you do, and all throughout the life of your company. Think about the impact that the COVID-19 crisis had on a company like Airbnb for example, or even Zoom, and how it impacted their apps’ features. And of course your company itself will change a lot over time, you will onboard more customers, with more diverse needs, expectations and tech-savvyness levels, you will scale up to integrate with third party systems like Customer-Relationship Management (CRM) platforms or customer support platforms, etc. And of course, the mobile ecosystem itself will evolve under your feet, what mobile devices can do, their technical characteristics, the technology used by developers to build those apps. All of those changes, whether they are internal or external will require a constant ongoing and recurring effort to update your application(s), some features will need to be re-developed because of tech changes, some features will need to be removed or added over time, your branding will probably also change every once in a while.

All of those changes are the reason why Pivoting is such a key concept, and why any business model reflection should start with clear values, a long term vision statement, and a strategic mission statement.

And that’s also why on the long run, most startups who have their own apps also have their own developers in house. Now even though it would be too big of a risk to try and hire a whole team of app developers right away, it’s important to look for partners that you can create a true partnership with, with a lot of trust, flexibility, expertise and initiative.

For example a lot of agencies will work on a fixed price contract: so they will ask you for a detailed specification beforehand, they will estimate a budget and a time frame for that specification, generally add some extra contingency in there, and start working on that. That generally means you won’t see anything while the app is being developed, you will have to wait for the end of the contract, or very late in the implementation to actually see a first version of your product (tunnel effect). And if you realized some important change was necessary in the specification, you will have to pay more to amend the specification and implement those changes. Some agencies go as far as to make you sign a seemingly very cheap fixed price contract at first, knowing full well that they will be able to make up for it in change requests later on, at which point you will be locked in to them, you will have no choice but to pay them to finish. Which kind of defeats the whole purpose of fixed price, right?

The fact of the matter is that fixed price contracts only ever make sense for well known and pretty stable app development projects, like an app for a restaurant or a hairdresser or a news platform, etc. In those cases, the feature list is pretty easy to estimate and pretty stable too. The thing is that in most of these, the needs are so generic that a simple no-code app will do the trick just fine. But fixed price is not realistic for a startup. You need to think in terms or time-and-material.

In this collaboration mode, whoever develops your app will invoice by the hour or by the day of work, and then it’s up to you to figure out how much of your monthly budget you can dedicate to building your app. The more you need to get done, the faster your need it, the more expensive it will get. But generally, this kind of collaboration is based on an Agile process that gives you a lot of visibility and power over the backlog of what needs to get built, the priorities that you determine, and it even gives you the possibility to modulate your budget based on your cash flow, or even to interrupt development if it’s not working. Then one of the questions becomes how much you will pay for each hour/day of work, and this can vary greatly. In Europe for example, we are generally talking about upwards of 50€/hour or 400€/day of development all the way to 100€/hour (or about 800€/day), sometimes more if there are intermediaries involved or a lot of corporate overhead. That depends on the expertise of the developers, how much of the whole development process they can take care of (more on that later), but more importantly their experience and how fast they work, and how much coordination and project management are needed. If you hire a 400€/day developer who invoices twice as many days and doubles your time-to-market compared to a 800€/day developer, the total cost of development may not vary that much, but it can have an important impact on your growth. Another important aspect is quality: a 400€/day developer will generally be a junior developer, with a small amount of experience, and can sometimes produce buggy code, that will require a lot of back-and-forth, and a code base that might be very hard to pass over to other developers later on, or to simply maintain on the long run. Clean code and good software architecture also make a huge difference when you want to add new features over time. Think of it like the difference between a house built by an experienced construction craftsman and one built by a rookie do-it-yourself amateur. This difference is generally very hard to evaluate for non-technical founders but it can truly make or break your business perspectives: look up Technical Debt. I won’t do it here, but I can tell you many stories of companies who took too many shortcuts, or didn’t invest enough in the right development partners early on, and then came very close to or downright failed because of it.

And then of course, there is the possibility of near-shoring (Eastern Europe, North Africa, etc.) or off-shoring (India, Pakistan, etc.) your development. Off-shoring is usually the cheapest option, near-shoring being the least damaging compromise, a little more expensive than off-shoring but less “remote” in all meanings of the term. To be completely honest with you, I personally think that off-shoring is almost a scam. Not necessarily in terms of technical expertise, as countries like India have very good education systems and very capable engineers. But there are things in their management culture (very little place for individual initiative), in their organization (a lot of turnover on projects), in the culture shift (their usage habits are simply different from a European’s) that generally more than make up for the apparent savings on daily rates. And I’m not even talking about the problems caused by timezone differences, language quirks, and again, “remoteness” in all possible ways. Most big companies who still work with offshore partners in India have some of their people there, on site, to manage and monitor projects and report back home, and otherwise need big support teams in house to specify everything in the most minute details so that Indian developers can then implement the specification without having to fill in any gap. It’s exhausting, and it’s expensive on the long term. So don’t do that.

Near-shoring is less of a stretch, the timezone difference is less of an issue, they are culturally closer to Europeans, so collaboration is generally smoother. And it is easier to find partners or agencies with a stronger sense of entrepreneurship and initiative who will be better at turning some of your vague ideas into actionable features without the need for bible-like specs. But still, the risk is higher, and I have seen more projects fail when working with Bulgarian, Polish or Romanian developers than with closer-to-home developers. Of course I would say that, I’m a freelance Belgian developer, and my daily rate is in the upper part of the ranges I gave earlier. But the one thing you should take away from this is to not just look at the daily or hourly rate. Think of the total-cost-of-ownership or your app. How long will it take to develop? How will it affect your time-to-market, both for the initial version of your application and for future features and changes? How will it affect the quality of the deliverables, and how your app will react to subsequent waves of new users? How easy will it be to grow the development team, and ultimately take over development entirely with your own people? These are all important factors to consider as a business owner, and there is no miracle: what you save one way, you generally end up paying for it another way.

What does your app need?

Mobile apps rarely work in a vacuum. Most apps work in conjunction with a backend of some sort. We will talk about different kinds of apps in the next section, but whatever the kinds of apps you need, they generally store their information in a database, and their files (pictures and other documents) on a file server of some sort. Plus, if your app processes payments, needs to receive push notifications, or integrates with third party services, all of those things are generally integrated on the backend. This server-side software is vital to most apps and its development, hosting and maintenance needs to be factored in to your app’s development time and budget.

Development of that kind of software generally involves a very different skillset than the one required to develop frontends like mobile apps or websites. And it’s rare to find those skills in the same person. There are even some development agencies who specialize more in frontend or backend development, and some frontend agencies specialize in either web or mobile development. Personally, I started my career as a backend and web developer, before extending my skill set in mobile development. So I can do it all which has a positive impact on efficiency and speed since there is less coordination needed between different actors, and more consistency between the different components of the overall software system. On the other hand, it’s true that more specialized developers can have more expertise with very specific aspects of backend or frontend development like:

  • security: highly critical datasets like financial or health data for example
  • compliance: certain specific regulations you need to abide by, like PSD2, etc.
  • performance: dealing with huge surges in usage like food orders, time-based sales, etc.
  • data-mining: big data analysis, machine learning, business intelligence, etc.
  • integration: Single-Sign On integration with corporate systems, Enterprise Resource Planning (ERP), Customer Relationship Management (CRM), etc.
  • 3D and gaming for frontend applications involving Augmented Reality, Virtual Reality or simulation
  • Media applications like sophisticated image, audio or video processing
  • And many more.

If your app involves one of these things, it might be interesting to look for partners who have specific experience in these domains.

But whatever the case, in addition to your app, you will need a backend to be developed, but also hosted, monitored and maintained. Hosting refers to making the backend available on a server somewhere, either in house (data center) or in the cloud (Amazon Web Services, Microsoft Azure, Google Cloud, etc.) so that apps and other frontends can access and share data and files over the internet. Monitoring involves setting up the right tools and processes so that if any of your backend components crashes or fails for whatever reason and becomes unavailable, your team is immediately notified and the situation can be remedied as soon as possible with a minimum loss of business. And maintenance implies regular security and performance updates to your backend software so that it remains available and doesn’t expose any of your customer data to your competitors, hackers or other unauthorized users. Sometimes, you will here those concepts referred to as DevOps, where Dev is the development of your backend and its databases, and Ops (or operations) is the hosting, monitoring and maintenance part.

To make things easier, there are a lot of Infrastructure-as-a-Service (IaaS) or Backend-as-a-Service (BaaS) solutions that automate a lot of those things. For example, I personally love Google Firebase, that relies on the infrastructure of Google Cloud. It is incredible versatile, easy to integrate and crazy cheap for startups.

But overall, the complexity of your data model, the criticality of the data you want to store and process, the number of third-party services you want to integrate with your services, all of those backend things will influence your app’s overall budget and time frame by a lot.

Last but not least, coming back to the app itself, you need to think about the device features you will need to integrate with:

  • Geolocation, whether it is one-off instant location, location tracking, navigation or even location-based notifications (geo-fencing), and at what level of accuracy (country, city, neighborhood, meter)
  • Camera, whether it is taking pictures or loading pictures from the photo library, taking videos (or loading them), recording audio, etc.
  • Phone data sources like the address book, the calendar, etc.
  • Bluetooth, whether it is connecting to your own devices, scanning for nearby Bluetooth beacons, etc.
  • More cutting-edge device-specific capabilities like Augmented Reality (AR), client-side machine learning, mobile assistant (Alexa, Google Home or Siri), etc.
  • Gyroscope, compass and other instruments built into devices
  • Map access and associated services like geocoding, reverse geocoding and so on
  • Push notifications, whether they are individual and transactional or more generic marketing announcements and offers
  • And some more.

Those mobile capability integrations have a big impact on the kind of tech stack that can be used (more on that later) and on the overall complexity of your application.

Who is the app for?

The most important aspect of your app is its users, and depending on who they are, how many of them there are, and how heterogeneous they are, it will have a big impact on your app, how long it takes and how much it costs to develop and maintain.

First off, what big categories of users will interact with your app. For example, if you are just selling your own products through your app, there is only one category of users: buyers. Or is there? Because you will probably need to update the information about your products, add new ones, etc. And you don’t want to call developers each time you want to modify data in your database. So most likely they will need to provide you with some form of backoffice, either mobile or more likely web-based, that will let you manage all your data. For more sophisticated multi-sided platforms like a food delivery company for example, you might have other users like couriers, restaurant staff and customer support people in addition to consumers themselves. Each of these categories of users will come with their own needs and features, and the more you try to cram all those features into the same app or frontend, the harder it will be to use. So you might need several apps, or an app that can easily adapt its interface to the kind of user using it. And one important instrument to think about all that is user personas: do you have a user persona for each of your user groups?

Another element to take into account is mobile platforms. There are mostly 2 dominating mobile platforms out there, Android and iOS, with different market shares depending on the country and specific demographics you want to address. If your customer target is mostly made up of CEO’s and high level executives in the US, you will likely have more iOS (iPhone) users than Android users. On the other hand, if you target third world countries, you will have more Android users, and not the most up-to-date kind. Figuring that out can have a huge impact on the complexity of your frontend. If you already have a first preliminary version of your product, analytics can give you numbers on that.

The smallest common denominator is generally a responsive website, meaning a website that adapts very well to all sorts of form factors, including vertical smartphones or all sizes. These responsive websites generally work on all devices, even the oldest ones, whether they use Android or iOS as their operating system, and they are easier to update and maintain since you don’t need to make them available through the Apple App Store or the Google Play Store. But they generally offer a sub-par user experience and limited integration with the device’s capabilities.

Then we get into the real app territory, and there too, there can be several approaches. The most natural one is what we call native apps, meaning apps (plural) developed specifically for each platform with the technologies provided for each one. Those native apps provide the best possible user experience, best performance and best integration with device capabilities like geolocation, camera, Bluetooth, etc. (more on that later). But they do so at the cost of having to develop and maintain two different applications that you will need to keep in sync. If you can find a developer that can develop both native Android and native iOS apps (like yours truly ?), that will be ideal in terms of coordination and consistency. But again that is rare, as most software engineers specialize in either one of these platforms. So this model is more adapted when your early users tend to use one of those two platforms, so that you can focus on building an app for it first, and then later expand your investment over to the other one when you are ready for it.

Finally, you can go for the hybrid app approach: there are technologies that make it possible to develop apps for both platforms but with just one code base, or at the very least with all the common code in one place, and a little bit of platform-specific code on the side. Those technologies are often third party, which means they are provided by other actors than Apple and Google, hence they need their own development tools and specific skill sets. Here are some big names you might hear about:

  • React Native is a technology provided by Facebook, that makes it possible to develop native apps for both Android and iOS using web skills like Javascript (probably the most popular right now).
  • Xamarin also makes it possible to develop native apps for both platforms but using Microsoft technologies (C#, .Net, etc.)
  • Flutter is a toolkit provided by Google that offers excellent performance (same as native), excellent developer experience and although it is the most cutting-edge of those three, it’s rapidly gaining in popularity (which personally pushed me towards it).

And there are other such platforms but I can’t cover them all here. The one thing to take away is that the choice of software strategy (what we sometimes refer to as “tech stack“) will greatly impact the user experience of your users, your time-to-market, the maintainability of the whole thing, how easy it will be to bring on more developers or hire your own one day. But the best tech stack is the one that your software developers master, so choose your partners wisely and let them work with the technologies they do best. And again, all of that depends on who your users are, and what sort of devices they use.

About that, it’s always important to think about form factors. Most mobile apps are primarily for smart phones, but if you simply use a smart phone app on a tablet like an iPad, the user experience will be awful. So even with a native or a hybrid app, you might want to think about responsiveness, which is how the application needs to adapt to bigger screens like tablets.

Last but not least, don’t forget to think about the languages your users understand. Internationalization is often an overlooked topic but it can also impact your software architecture and the cost of your apps. The easiest thing is to internationalize your app’s user interface, meaning all the buttons and generic elements of it. But if you also need your data to be internationalized, like restaurant menus for a food delivery app, or product listings for an ecommerce app for example, then it’s a whole different story. Plus, some languages read right-to-left instead of left-to-right, or use other character sets like Chinese or Russian, and this can also greatly impact the design of your app. Whatever the case, you will need tools, processes and resources to translate your app’s interface so that it remains usable for all your users. And even if your app only targets French people in France for now for example, you might want to prepare internationalization right now for the day when you want to grow your business outside of those borders, knowing that it can be much more expensive to add that internationalization after the fact.

How will your app make you money?

In general, your app will at least be part of your sales channel, which mean you will use it to sell goods or services. And there are many different ways to monetize your app.

The most obvious revenue model is to sell your app directly, which you can easily do for native or hybrid apps you sell through the App Store or the Play Store. In that case, you don’t sell your app directly, Apple or Google sells it on your behalf, and pays you royalties after taking a commission. This commission used to be a hard 30% for both platforms, knowing that you could still offer your app through another channel like a website on Android (side loading), but you can’t do that for iPhones. These 30% might seem harsh, but remember that it includes payment processing fees (generally around 3% anyway), hosting and promotion of your app on the store, access to platform services like push notifications, etc. And of course this lets Apple and Google finance the development of all the tools and technologies used by developers to create your beautiful app. Plus, as I said, it used to be 30%, but recently, Apple announced a small business program that makes it possible for small businesses and startups with less than 1 million dollars in annual revenue to take advantage of a 15% commission instead of 30%. You can already enroll today, but the program will only be launched on January 1st 2021. And it would not be surprising to see Google offer something similar soon. But there is a problem with this revenue model: it creates a huge barrier to entry: only a small proportion of mobile users are ready to buy mobile apps outright, and they have to be convinced of its value before they hit the purchase button.

Which is why another model has become extremely popular over the years: in-app purchase. With in-app purchases, you can offer your app for free, and then offer services and feature access inside your app for a fee. That fee will be submitted to the same 30% (or 15% for small businesses on iOS) as if you sold your app outright, but those in-app purchases will go through the same painless payment process with the card or Paypal account associated with the user’s App Store or Play Store account. There are two important limitations though:

  • If you are developing a Business-to-Business application, selling your services to companies instead of end consumers, those companies generally require business invoices and payment channels (check, on-invoice, etc.) that are not supported by the heavily B2C-biased model of stores. The good news is that Apple has an exception in their conditions just for that use case that lets you offer other payment channels in addition to in-app purchase. Unfortunately, it doesn’t look like the Play Store has similar exceptions yet.
  • In-app purchases are only for virtual goods and services that users can access within the app. If you are using your app to sell physical goods to be consumed in the real world for example, you must use other payment channels. And integrating a payment processor like Stripe, Paypal, Adyen or similar also has a cost per transaction in addition to the development cost to integrate them into your app and backend.

And then of course, beyond those payment channel considerations, there is the question of the revenue model itself:

  • Will you take a commission on each transaction where your user buys something through the app?
  • Will you have a recurring subscription model where depending on the subscription tier they are on, your users will have access to different features?
  • What happens when a user wants to cancel their subscription, or revert a transaction?

All those questions and many more can have an impact on your app’s design and complexity.

Last but not least, if your revenue model is based on a lot of micro-transactions, you absolutely need to talk with your accountant about it, because most accountants will charge you for their services based on the number of transactions you process and they have to account for. Some accountants and fiduciary firms offer automation and integration services so that your transactions can be reported directly to your books, but they are not always ready for it, and it can incur additional costs.

Do you have a lawyer?

Probably not. But you might need one. The moment you start gathering identifying information about your users, say hello to privacy policies, cookie policies and other terms of service, not to mention the new kid on the block: General Data Protection Regulations, aka GDPR.

It’s another thing that is often overlooked, but it’s not enough to know about it and add a GDPR logo to your website to be compliant. There are roles to define, rules to understand, tools and processes to set up to keep track of which data you store and why, and which third party providers have access to your users’ data.

And just like your app itself, those things will likely evolve throughout the life of your business, need some updates and so on.

In addition to that, for everything you contract other companies or providers to do for you, you will need to have or review contracts, which are especially critical when we are talking about software consulting contracts.

For all of these, having a lawyer who knows your business and follows you can be precious. And even if you will not need them a lot at the beginning, even if you can use generic services like Iubenda to take care of some of your legal needs, it’s generally a good idea to think about those things right from the start, because they have an impact on where your data can be stored, how it can be backed up, what you can legally do with it and so on.

What will your app look like?

I kept this one for the end because it is the most impacted by all the previous ones, and the most crucial resource that any developer will need to answer the dreaded question: HOW MUCH?

There are 3 main aspects here: User Experience (UX), branding and User Interface (UI) design.

User experience is all about the general flow of screens and actions inside your applications. How will features be organized? What are the key use cases you want to cover, and how will they work? Which ones are absolutely essential in the first version of your app versus which ones can be added later, and in what order of priority. The best way to formalize user experience design is by drafting wireframes, rough drawings of the different screens of your apps and the interactions between them. This can help developers determine how many screens your app will need, and how complex they will be, which is essential for any kind of estimation work. At this stage, no need for any fancy colors or branding, and it is even possible to turn these wireframes into a working prototype that can be played with without any coding involved. Some developers (like myself, of course) can help you extract those wireframes from your ideas.

The second important aspect that can be integrated into your app to make it feel less generic is branding. Do you already have your own visual identity, color palette, fonts, icons and logo that can be turned into some sort of graphic charter or theme for your app?

And of course the finest of the finest of design is User Interface (UI) design. That’s when you start adding more custom elements and interactions to your app, that require more work from developers but make it look more specific to your business. I generally advise entrepreneurs to keep that for a later version, because it can add a lot of overhead to development, depending on which tech stack is being used (for example, UI design is much easier to customize in Flutter than in almost any other tech stack, except maybe for responsive web apps). Plus, it’s best to wait until the user experience of your app has stabilized a little bit before you start adding bells and whistles. So unless visual design is part of your core offering, or creates a huge competitive advantage for you, it’s best to focus on UX and branding first.

Also, keep in mind that Android and iOS each have their own UX and UI idiosyncrasies and customs, and it can sometimes be better to stick to the design conventions of each platform rather than having the same exact experience across platform. Always remember that as a business owner, you are managing one app on several platforms, but your users will use many apps on one platform, and the more it fits into the ecosystem they are used to, the more comfortable it will be for them.

This design work, whether it is UX, branding or UI, generally requires yet another skill set, some agencies offering this service too, others specializing in design only. Personally, I can generally help entrepreneurs with UX design, turning vague ideas into wireframes, and then those wireframes into a usable prototype. But for everything branding and UI, I generally work with other designers.

But whatever the case, a set a wireframes is absolutely required to get any sort of estimation about how much an app development will cost, and how long it will take. Don’t trust any company, developer or agency who gives you a budget or a time frame number without any wireframe or detailed idea of what an app will look like. It’s a TRAAAAAP!

Epilogue: are you a product manager?

Knowing what you need to put in your first version, and being able to express it is only the first step. Through the process of formalizing your ideas into wireframes though, you will no doubt come to realize that designing a product is harder than you imagined. Every usage scenario will come with edge cases, potential errors (either from the user or from the app/backend).

What happens if a users tries to sign up with an email address that’s already registered? What happens if the users kills the app in the middle of a long workflow? What happens when the user taps a push notification?

And since, as we said it before, developing such an app is not a one-shot thing, you will have questions like that all the time. If your developers are really good and have a product sense, they will take initiatives to fill in those gaps, but these developer-minded solutions might not be what you need. And if you always wait for developers to come back to you with these questions as they are raised during development because you hadn’t anticipated them in the first place, it can slow down development considerably, and kill your own productivity with all the interruptions.

That’s exactly where Product Managers, or Product Owners come into play. It takes a special kind of experience and mindset to think about all of this, to anticipate all these edge cases, to think of design solutions that remain consistent across your entire product.

And as time goes, you will need to maintain several versions of your app in parallel, the one that’s currently in the stores, the one that is currently being tested, and the one that developers are working on. And each version of the app will need to be coordinated with the proper version of the backend. All of this can quickly become a mess.

Not to mention the issues with coordinating more and more actors as your company grows: designers, developers, testers, third-party providers, lawyers, all those people will have to work in unison and in a coordinated fashion if you want your product to move forward. And again, you might think that you can do it all today, but wait until all of this falls on your lap while you are fundraising, recruiting, dealing with the administration, answering interviews and so on.

Last but not least, as your app becomes more and more sophisticated, you will face situations where there are several ways of doing things, and where the “right” way won’t be obvious from the get go, especially if you have to think about the habits of your users, the conventions of the platform they are using, the way things are done throughout the rest of your app, what is technically reasonable to implement, what is legally advisable, etc. To find the right approach, you will need to run experiments, to do some A/B testing, to talk to several of the people involved in this feature, etc. And again, that’s where a good Product Manager can literally save your product.

So in a first approach, maybe you will be able to do it yourself, or share this responsibility with your developer or your agency if they have this product sense. But given the time it takes (I have seen some founders having to dedicate 80% of their time to this), I can only recommend that you start planning this as a recruitment as soon as possible, because managing a product, especially an app product, is a job in and of itself, and a hard one.

Conclusion

Remember that developing an app is not like building a house or a car. It is a highly unpredictable process, that involves a lot of craftsmanship, expertise, trial-and-error, exploration and uncertainty. So always remember that it’s more an investment than a cost: an intuitive, well designed and well developed app can be a very important element of your business model, but a cheaply executed one can also break your business. The saying goes that the most accurate estimation for a software project can only be produced once software has been delivered. This is what we call the cone of uncertainty. So however stressful it can seem to have only vague numbers at first, try and embrace it and have a software development process that is agile and flexible, that allows for changes and evolve based on what you learn on the ground, and try to find partners who will take initiative, anticipate your concerns, be reliable in terms of quality and efficiency and will strike a balance between engineering and pragmatism.

If you want the short version of this brief, here is a link to a document you can work with.

And of course, if you need an app developed, or a guide to help you find answers to all those questions, you know where to reach me.

Couchbase and the API model

On my last job, working mostly on our iOS app as a frontend developer, something struck me hard. So much effort went into building plumbing code, things like offline caching, pagination, connection error handling. Whenever I spend so much time on infrastructure code instead of implementing features, I’ve always started questioning the very paradigms I’m working with. Back in the days when most of my effort was spent on configuring Spring and Hibernate components, I discovered Model-Driven Architecture and the ability to generate most of this plumbing out of UML in order to focus on the functional aspects of the software I wrote. But I still had to sift through all this generated code, customize it, understand all the complexity of it, until I came across Groovy and Grails, and their amazing expressiveness. With less verbose code, the path from functional specification to working feature was shorter and shorter. But then I moved to the iOS frontend, a frontend where neither code generation nor language expressiveness would help me (for the latter, it all changed with Swift of course, but that’s for another story). The more I had to fight those mobile constraints, the more it became clear that we were trying to fit a round peg in a square hole.

The web model is based on a few assumptions. When a computer connects to the internet, it’s basically a relatively dumb terminal with very little processing power, at least inside the browser since you can’t access the entire processing power of the machine it’s running on, for both security and genericity reasons. It also has very limited storage space, mostly for security reasons I guess. But the one thing you can count on is permanent connectivity. And nowadays, you can even count on a pretty good bandwidth too. Based on my understanding, the very paradigms the web runs on are tailored to this state of things. The HTTP protocol is designed to let you access Resources, those resources being online always in reach, at a specific Unified Resource Locator (URL). If you need a resource, simply GET it from the network and it shall be returned to you. And every time you request it, you will get it consistently, in its latest known state, wherever you request it from. So no need to store it. No need to do some complex processing with it either. Actually, it’s better if you let the almighty Server to it for you. He has more power, he centralizes all of that. All you have to do is request and display. Now of course, for performance reasons, you might have to keep a hold of the resource and use tricks to know whether your version is up to date, but clearly the surest way is just to request the resource again, and since the network is always there with its wide bandwidth, why bother. The entire concept of RESTful API’s seems to be based on those assumptions. And that’s what we’ve been using to connect web frontends to their centralized backends for decades.

Now if you look at the mobile space, and especially at the native app space, it looks like all the constraints are all reversed. The terminal is certainly not dumb, you have access to quite some storage space and native processing power, but connectivity on the other end, is far from granted. It can be very slow, it can even drop on you and come back without prior notice. And yet, despite all those reversed constraints, for the most part, we are still using RESTful API’s to connect mobile native apps to server backends in order to share data amongst users. We’ve always done it that way, right? Why change?

You might be offline from time to time? OK, just build some local database to cache data then, and then serialize all your JSON coming from your RESTful API into your mobile database format, whatever it might be.

Your bandwidth might be narrow, and you can’t download too much information at a time without impacting the reactivity of your app (and believe it or not, mobile app users are far less enclined to wait than website consumers)? No problem: just build pagination into your API and into your frontend and only request the information you can display, even if that makes your caching much more complicated to implement.

At some point, you realize that maintaining all those different layers of data representation (database storage, backend objects, API resources, mobile database storage, etc.) and building pagination throughout your entire app is taking up more and more time. It’s increasing your time-to-market, crippling your user experience and decreasing the quality of your code (more code, more bugs).

For a time, you think there’s no way around it. You look at others and all mobile developers seem to be struggling with the same issues. Hundreds of Github frameworks and libraries are here to help you with them.

But I’ve always been a gotta-be-a-better-way kind of guy. So I started digging into alternatives. I first discovered Firebase (recently acquired by Google, how about that for a sign?), and this amazing idea of doing everything locally in the frontend, and then let the infrastructure handle the data sharing with all the instances of the app. It was appealing, but I didn’t like the fact that I had to hand over my data to a third party for eternity: I was OK to host it in the cloud for some time, but at some point I wanted to be able to take it back and host it on my own infrastructure. Plus, I never got it to work in my simple iOS testing apps.

And then I came across Couchbase Sync Gateway. Couchbase is mostly known as a NoSQL Big Data store, uniting the best of CouchDB and Membase. But about a year ago, they added another ingredient to the mix, in the form of 2 new components:

  • Couchbase Lite lets you store data natively in an iOS or Android app in a NoSQL way, which is already amazing in and of itself. No more serializing JSON data into Core Data objects or SQLite records.
  • Couchbase Sync Gateway, which lets you automagically synchronize a Couchbase Lite instance with a server-hosted Couchbase server database.

So all of a sudden, with a Couchbase architecture you can centralize and share data between remote clients without having to implement a complex RESTful API with caching and pagination and all that. All you have to do is store data in your mobile data store, and let Sync Gateway handle the synchronization, signal synchronization conflicts and let you handle them, and even deal with data access control in a clever way. Less time spent on writing code (or maintaining libraries) to handle network connections and API requests, more time spent on real value-adding features. And you even get some nice additions for free, like the ability for your users to use your app even when they’re offline, and then let sync gateway share it automatically with the rest of the world when the network comes back.

For months, I was amazed by this amazing prospect and the incredible powerful mind shift it creates: I finally get to use all the storage and processing power I have at my disposal, and I can finally embrace the fragility of my network connection instead of trying to fight it.

But over the past few weeks, I’ve discovered the catch: although amazing, the Couchbase technology is still very immature. Not technologically speaking. I have the utmost respect for the engineering teams there, who are doing an outstanding jobs coming up with such breakthrough clever ways to change the way we think our mobile architectures. But on the market side of things, I’m still struggling. Couchbase is obviously coming from a very special world, a world where big companies are spending (and making) billions by storing, processing, analyzing and repurposing huge amounts of data. And then they added data sync and mobile into the mix, but my impression is that they did that as an afterthought, as a bonus for big data companies. I’m a startup entrepreneur, I don’t have that kind of data to manage (not yet) and I certainly don’t have the funding that goes with it. I just want to build a mobile app in the easiest and most efficient way, and still think ahead in terms of performance, so that when I do start to have traction, my architecture can scale effortlessly. Technically, that’s exactly what Couchbase holds. It certainly has all the components to do it. But then when you start trying to set up a production-ready infrastructure to host that architecture, that’s when things get tough:

  • There is no cloud-based offering to let you get started with Couchbase in a couple of hours and focus on the development part without having to worry about the operations part. There have been some attempts, like Kurobase and CouchSyncHQ, but for some reason they both disappeared overnight, and I haven’t been able to figure out why.
  • So the only way is to set it up yourself. Couchbase provides you with some Amazon Web Services AMIs, but the Sync Gateway ones seem completely outdated, and finding documentation about these is hard. Plus, they make for a very expensive setup to run.
  • Recently, I’ve been working a lot with Traun Leyden, trying to help him improve his AWS+Docker+CoreOS+Couchbase cluster setup and it is easy to deploy now, but not particularly easy to customize, especially when you’re not a ops guy and you don’t understand all these technologies very well.
  • So I tried contacting Couchbase themselves, asking for a consultant to help me set up a basic cluster, and I got a quote for 4 days of work, at 2500 bucks a day. 10k is certainly not a reasonable amount to spend on infrastructure when you are a non-funded startup, let alone for a side-project. At the very least, that’s a very tough mountain to climb at first, and that kind of service offering is clearly more adapted to the usual big data customers of Couchbase than to me.

Today, I’m stuck on this tough choice: should I give up on this amazingly promising technology to switch back to a more pragmatic, easier to deploy and maintain architecture, at the potential long term cost of adding more complexity to my app? Or should I stick with it and find a way to customize it, possibly outspending what it would cost me to ask one of their consultants to do it for me? I really don’t know, and I wish Couchbase allocated more resource to making it all more accessible to small entrepreneurs who just want to ship a working app in no time, and improve it very fast. Maybe we should be the ones building that layer on top, I don’t know yet if I want to shift focus like that. Just thinking out loud here. If anyone from Couchbase wants to comment, I’d love to talk about it.

Android Studio and the future of Java IDE’s

Yesterday, when they announced Android Studio during the keynote of Google I/O, my very first reaction was “Oh no! Too bad for Jetbrains! Even though they build awesome plugins for Android development, they will be taken over by Android Studio, beaten by their own platform!”. But then I saw this:

And I realized that even though they will probably have to sit on months of in-house development, so many Android developers will get used to the amazing productivity of the IntelliJ platform, realize how much Eclipse sucks as an IDE, and switch to IntelliJ IDEA as their primary Java IDE. Even more so as I expect other platform providers to switch now that Google has opened the way. Adobe? SpringSource? Are you listening?

But there is one question that still bothers me: mobile development very often requires backend development as well. So will Android Studio be available as a plugin for IntelliJ IDEA or will I have to keep 2 IntelliJ-based IDE’s opened at all times? Because if that’s the case, it will suck productivity-wise and I’ll have to upgrade my memory.

UPDATE: And… all my questions are now answered! Good job guys!

Payment Processing Landscape in Europe

It is a well-known yet long-standing fact that setting up payment processing is much harder in Europe than it is in the United States for example. A major reason for that complexity comes from the fact that, contrary to the US where they have big national banks that cover the entire American market, European payment processors have to deal with 27 different legal and tax environments, and very often 27 different sets of partnering banks. That’s probably why it took so long for Braintree Payments and Stripe to cross the Atlantic ocean, and even as they do, they do it in a very limited way.

I’m currently working on a new experiment for which I want to process payments in a native iOS app without disturbing the user experience I have spent hours perfecting. So I reviewed the most well-known contenders, limiting myself to the most modern and country-agnostic actors, and I’ve completed the following benchmark which might be useful to some of you.

Note that I have a few special requirements that are decisive for me but might not be applicable to you:

  • I’m using Parse for my backend, which doesn’t allow me to run any server-side code easily. I can run simple Javascript code that can call REST API’s (or special integration code when they have a partnership like with Stripe).
  • My service has a peer-to-peer marketplace business model, which means that I take a commission on direct transactions between end-users, which payment processors call Third Party Payments Aggregation. For acquiring banks, this kind of business models is often considered a risky one because it poses all sorts of questions in case of refunds. So not all acquiring banks accept TPPA.
  • A lot of payment processors say they offer mobile processing services when in fact they mean mobile WEB payment, which means stripped-down restyled version of their web forms that you have to integrate with web views and greatly disturb the user experience. That may be enough for most applications, but I didn’t want to sacrifice user experience (and thus conversion) in my case.
  • As I said, my app is just an experiment for now. I don’t know how much volume it’s gonna deal with, and I don’t want to wait months to have my payment processing ready just because I’m a “small fish”.
  • Since it’s a mobile app, I would really like to be able to store credit card data for later so that my end users don’t have to enter 23 digits for every transaction (secured vault).

Also, be careful. This is the situation as of mid-April 2013 and it’s very likely to evolve a lot over the coming months and years.

 

  • I’ve rejected Paypal and Adyen because of their lack of native iOS integration
  • Stripe is not available in Belgium yet and will probably not be anytime soon
  • Braintree Payments is available in Belgium but their mobile wallet (Venmo Touch) is not available in Europe yet, and the dealbreaker for me was that they couldn’t find any acquiring partner in Europe to support TPPA.
  • Ogone is very cumbersome to set up, their documentation is awful, their backoffice interface is simply unreadable, but more importantly they don’t support native iOS integration.

A few days ago, Paymill released the first beta of their iOS and Android SDK. The activation process went pretty smoothly and their documentation is rather clear compared to some of the others. They are using Acceptance as an acquirer in Germany. I have started to integrate the SDK into my app and so far it works great. The API is pretty straightforward, even if they still need to work on naming conventions in my opinion. But it’s definitely the most promising actor on the market today.

If you have any complement to add to this information, feel free to leave a comment.

EDIT: Thanks to marcos for pointing me to 2 mistakes in my benchmark. Paypal actually does have a REST API but it’s only available in the US for now (in beta). They also have a native iOS SDK that I hadn’t found. Given that the REST API is not available in Belgium, I wouldn’t be able to verify payments, and I still don’t trust Paypal to keep my funds available so I’m still sticking with Paymill for now. But I have updated the benchmark accordingly.

EDIT2: There’s a new kid in town. Leetchi just publicly announced their new e-money service for collaborative consumption and marketplaces: MangoPay. I’ve added them the the benchmark. Their REST API is still a little too low level for me and they don’t seem to be planning an iOS or Android SDK right now so it won’t make me switch from Paymill yet. Plus their e-money concept needs some explaining. But it looks like an interesting European alternative to Stripe Connect.

UPDATE: Paymill just released their marketplace offering to compete with Stripe’s Connect: https://www.paymill.com/en-gb/product/unite/

UPDATE: Braintree Payments just released their marketplace offering, but it’s only available for US businesses for now: https://www.braintreepayments.com/marketplace

Top 5 reasons why you should consider Groovy and Grails for your enterprise software architecture right now

I’m so amazed when I see how so few companies are using Groovy and Grails right now, and are still using old stuff like Spring and Hibernate, that I thought I would jump in and do my share of educating. And why not give in to the fashion of top lists while I’m at it? So here it goes: if you are an enterprise software architect and you have a lot of Java in your world, you might want to read carefully what follows.

Why I switched back from Heroku to CloudBees

I used to have several grails applications deployed on CloudBees. I liked the fact that they were Java all along, I liked the smooth integration between Jenkins CI and the deployment environment. I really liked the fact that you could hide an application behind a username and password during testing. I just hated their design (seriously guys, hire a good designer) and I was not thrilled by their catalog of third-party services. So when Heroku announced that they supported Java applications, and then Grails applications, it was not long before I migrated all my apps over to their servers.

But more recently, I’ve had issues with a more plugin-rich application. And tonight, after several weeks of fighting, I migrated this app to CloudBees for one general reason: Heroku was really designed with RoR in mind, and even though they made a new stack for java apps, the old rules still apply:

  • if an application takes more than 60 seconds to boot, it is crashed. There is no way to adjust this timeout, and we all know that it can be pretty common for a java app to take a little more than that.
  • another consequence of this slow boot intolerance is that an application is automatically put in sleep mode after several minutes of not being used. Consequence: when someone comes to the site after such a period, say, in the morning, the app is rebooted out of sleep mode… and sometimes it crashes again. Terrible for availability. And apparently, even if you pay, there is no way to prevent that sleep behavior and to keep your app alive all the time. Actually to be fair, there is a way to disable idling: once you scale up to 2 dynos, idling is disabled.
  • since your app can be sleeping, implementing long-running background processes is very complicated too. You have to use their worker processes, but there is no documentation on how to do that in a java application, let alone in a grails app.
  • last but not least, even though I tried to limit the amount of memory of my app, it kept going over 512MB, which is the absolute limit. Once again no way to change that. It doesn’t crash my app, but it clutters my log with plenty of annoying warning messages.

That plus the fact that they refused to answer my last support request about memory consumption and marked it as solved when it got hard. The fact that they don’t have an easy continuous integration feature was not a deal breaker, but it adds to the rest.

Now I don’t know if CloudBees will be better on all these points, but it looks good on paper. Unfortunately, we java devs don’t have a lot of choice when it comes to cloud deployment. CloudFoundry is way too low-level, AppFog is still in private beta, and Amazon Elastic Beanstalk is awful for deployment (40MB take a long time to upload). What other options do we have that I’ve never heard of?

 

Grails, Vaadin and Spring Security Core

I got kind of bored with Flex and all the complexity it introduces by forcing you to switch between ActionScript and whatever you are using for the backend (Groovy in my case). I also got bored with having to regenerate my data service stubs on each server-side change, and having to handle the asynchronous remoting. So I started to have a look at Vaadin.

Vaadin offers the same richness of components as Flex, but I can code my UI with Groovy and it completely removes the need to bother about remoting and all that stuff. It’s really like my old Swing days and I love it.

Last week-end, I tried their AddressBook tutorial, and I adapted it to Grails using the Grails-Vaadin plugin. Then I modified the sample so that it uses GORM to store contacts. And finally I installed spring-security-core plugin to secure my business services with @Secured annotations. And it worked absolutely great.

I just released a new version of the Grails-Vaadin plugin with Vaadin upgraded to 6.5.1 (the latest version at this point), and I uploaded my version of addressbook to GitHub.

For me, the most interesting part is how I got security to work. All I had to do was to install spring-security-core plugin into grails and then define a simple SecurityService like the following:

package org.epseelon.addressbook.business

import org.springframework.security.core.context.SecurityContextHolder as SCH
import org.springframework.security.authentication.BadCredentialsException
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken

class SecurityService {

    static transactional = true

    def springSecurityService
    def authenticationManager

    void signIn(String username, String password) {
        try {
            def authentication = new UsernamePasswordAuthenticationToken(username, password)
            SCH.context.authentication = authenticationManager.authenticate(authentication)
        } catch (BadCredentialsException e) {
            throw new SecurityException("Invalid username/password")
        }
    }

    void signOut(){
        SCH.context.authentication = null
    }

    boolean isSignedIn(){
        return springSecurityService.isLoggedIn()
    }
}

Then I injected this SecurityService into my AddressBookApplication and used it:

class AddressBookApplication extends Application {
    private SecurityService security = (SecurityService)getBean(SecurityService)

    [...]

    boolean login(String username, String password) {
        try {
            security.signIn(username, password)
            refreshToolbar()
            return true
        } catch (SecurityServiceException e) {
            getMainWindow().showNotification(e.message, Notification.TYPE_ERROR_MESSAGE);
            return false
        }
    }
}

Then whenever I try to call a @Secured method:

package org.epseelon.addressbook.business

import org.epseelon.addressbook.dto.PersonListItem
import org.epseelon.addressbook.domain.Person
import grails.plugins.springsecurity.Secured

class PersonService {

    static transactional = true

    [...]

    @Secured(["ROLE_USER"])
    PersonListItem updatePerson(PersonListItem item) {
        Person p = Person.get(item.id)
        if(p){
            p.firstName = item.firstName
            p.lastName = item.lastName
            p.email = item.email
            p.phoneNumber = item.phoneNumber
            p.streetAddress = item.streetAddress
            p.postalCode = item.postalCode
            p.city = item.city
            p.save()

            return new PersonListItem(
                firstName: p.firstName,
                lastName: p.lastName,
                email: p.email,
                phoneNumber: p.phoneNumber,
                streetAddress: p.streetAddress,
                postalCode: p.postalCode,
                city: p.city
            )
        }
        return null
    }
}

If I’m not logged in as a user, I get an “access denied” exception:

package org.epseelon.addressbook.presentation.data

import com.vaadin.data.util.BeanItemContainer
import org.epseelon.addressbook.dto.PersonListItem
import org.epseelon.addressbook.business.PersonService
import com.vaadin.data.util.BeanItem
import com.vaadin.ui.Window.Notification
import org.epseelon.addressbook.presentation.AddressBookApplication

/**
 *
 * @author sarbogast
 * @version 19/02/11, 11:12
 */
class PersonContainer extends BeanItemContainer<PersonListItem> implements Serializable {
    [...]
    boolean updateItem(Object itemId) {
        try {
            personService.updatePerson((PersonListItem) itemId)
            return true
        } catch (Exception e) {
            AddressBookApplication.application.getMainWindow().showNotification(
                    e.message,
                    Notification.TYPE_ERROR_MESSAGE
            );
            return false
        }
    }
}

To see what it looks like, all you have to do is to download the code from GitHub, and run “grails run-app” at the root of it.
If you try to create a new contact of edit an existing one and save it without being logged in, you get an “access denied” message. But if you login as ramon/password, it works.

Note that this project uses Grails 1.3.6 but the plugin supports any version of Grails above 3.2 included. As always, your feedback is more than welcome.

My Case for DTO’s

In many of my posts about Grails and Flex integration, I take for granted that I use Data Transfer Objects to transfer data between my Grails backend and my Flex frontend. Put simply, Data Transfer Object are pure data containing classes different from the domain entity classes used to store data in the backend. I take it for granted because I’m deeply convinced that it’s the best way to do things and so far, experience has never proved me wrong. But I often get this question in comments or by mail (this is for you Martijn): why bother create an entirely separate class structure and copy data from entities to DTO’s and back instead of just using entities?

I’ve expressed my arguments a couple of times across various posts but I thought it would be nice to sum things up in here for future reference.

Where does it come from?

When I first started to work on enterprise applications and ORM-based architectures, it was with a Model-Driven Architecture framework called AndroMDA. AndroMDA was absolutely key in helping me getting started with Spring and Hibernate and I was especially inspired by one paragraph in their “getting started” tutorial, which I quote here:

Data Propagation Between Layers

In addition to the concepts discussed previously, it is important to understand how data propagates between various layers of an application. Follow along the diagram above as we start from the bottom up.

As you know, relational databases store data as records in tables. The data access layer fetches these records from the database and transforms them into objects that represent entities in the business domain. Hence, these objects are called business entities.

Going one level up, the data access layer passes the entities to the business layer where business logic is performed.

The last thing to discuss is the propagation of data between the business layer and the presentation layer, for which there are two schools of thought. Some people recommend that the presentation layer should be given direct access to business entities. Others recommend just the opposite, i.e. business entities should be off limits to the presentation layer and that the business layer should package necessary information into so-called “value objects” and transfer these value objects to the presentation layer. Let’s look at the pros and cons of these two approaches.

The first approach (entities only, no value objects) is simpler to implement. You do not have to create value objects or write any code to transfer information between entities and value objects. In fact, this approach will probably work well for simple, small applications where the the presentation layer and the service layer run on the same machine. However, this approach does not scale well for larger and more complex applications. Here’s why:

  • Business logic is no longer contained in the business layer. It is tempting to freely manipulate entities in the presentation layer and thus spread the business logic in multiple places — definitely a maintenance nightmare. In case there are multiple front-ends to a service, business logic must be duplicated in all these front-ends. In addition, there is no protection against the presentation layer corrupting the entities – intentionally or unintentionally!
  • When the presentation layer is running on a different machine (as in the case of a rich client), it is very inefficient to serialize a whole network of entities and send it across the wire. Take the example of showing a list of orders to the user. In this scenario, you really don’t need to transfer the gory details of every order to the client application. All you need is perhaps the order number, order date and total amount for each order. If the user later wishes to see the details of a specific order, you can always serialize that entire order and send it across the wire.
  • Passing real entities to the client may pose a security risk. Do you want the client application to have access to the salary information inside the Employee object or your profit margins inside the Order object?

Value objects provide a solution for all these problems. Yes, they require you to write a little extra code; but in return, you get a bullet-proof business layer that communicates efficiently with the presentation layer. You can think of a value object as a controlled view into one or more entities relevant to your client application. Note that AndroMDA provides some basic support for translation between entities and value objects, as you will see in the tutorial.

Because of this paragraph, I started writing all my business services with only data transfer objects (what they call “value objects”) as input and output. And it worked great. Yes it did require a little bit of coding, especially as I had not discovered Groovy yet, but it was worth the time, for all the following reasons.

The conceptual argument: presentation/storage impedance mismatch

Object-relational mapping is what Joel Spolsky calls a “Leaky Abstraction“. It’s supposed to hide away the fact that your business entities are in fact stored in a relational database, but it forces you to do all sorts of choices because of that very fact. You have to save data in a certain order in order not to break certain integrity constraints, certain patterns are to be avoided for better query performance, and so on and so forth. So whether we like it or not, our domain model is filled with “relational choices”.

Now the way data is presented involves a whole different set of constraints. Data is very often presented in a master/detail format, which means you first display a list of items, with only a few fields for each item, and possible some of those fields are calculated based on data that is stored in the database. For example, you may store a country code in your database, but you will display the full country name in the list. And then when the user double-clicks an item, he can see all the fields for that item. This pattern is totally different from how you actually store the data.

So even though some of the fields in your DTO’s will be mere copies of their counterparts in the entity, that’s only true for simple String-typed fields. As soon as you start dealing with dates, formatted floats or enum codes, there is some transformation involved, and doing all that transformation on the client-side is not always the best option, especially when you have several user interfaces on top of your backend (a Flex app and an iPhone app for example), in which case you’re better off doing most of these transformations on the server.

In anyway, if you change the way you store data, it should not influence too much the way you present the same data, and vice-versa. This decoupling is very important for me.

The bandwidth argument: load just the data you need

In the master/data use case, when you display the list of items, you just need a subset of the fields from your entities, not all of them. And even though you’re using Hibernate on the backend with lazy-loading enabled, fields are still initialized and transferred over the wire. So if you use entity classes for data transfer, you will end up transferring a whole bunch of data that may never be used. Now it might not be very important for hundreds of records, but it starts being a problem with thousands of records, especially when there is some parsing involved. The less data you transfer the better.

The security argument: show only the data you want to show

Let’s say you’re displaying a list of users, and in the database, each user has a credit card number. Now of course when you display a list of users, you might not want everyone to see the list of credit card numbers. You might want to expose this data only in detail view for certain users with certain privileges. DTO’s allow you to tailor your API to expose just the data you need.

The error-prone argument: argh! Yet another LazyInitializationException!

Of course there are associations between your business entities, and by default, those associations are lazy-loaded, which means they are not initialized until you actually query them. So if you just load a bunch of instances from your entity manager and send them over to your client, the client might end up with null collections. Now of course you can always pay attention, or use some tricks to initialize associations up to a certain level before you send your data, but this process is not automatic and it’s very error-prone. As for using things like dpHibernate, I think it just adds too much complexity and uncontrolled server requests.

The laziness argument: Come on! It’s not that hard!

I think that most of the time, the real reason why people don’t want to use DTO’s is because they’re lazy. Creating new classes, maintaining code that does “almost” the same as existing code, adding some code to service implementation to copy data back and forth, all of that takes time and effort. But laziness has never been a good reason for ditching a design pattern altogether. Yes, sometimes, best practices force us to do more stuff for the sake of maintainability and robustness of our code, and for me the solution is certainly not to shortcut the whole practice, but just to find the best tools to minimize the added work. With its property support and collection closures, Groovy makes both creating, maintaining and feeding DTO’s as simple and fast as it can be. AndroMDA had converters. There are even some DTO-mapping frameworks like Dozer to help you. No excuse for laziness.

For me, all the reasons above largely overcome the added work to maintain a parallel DTO structure.

Now of course, this is a very opinionated topic and you will probably have a different view. So all your comments are welcome as long as they remain constructive and argumented.

Flex on Grails, Take 2: Part 3

At the end of the second article in this series, we ended up with a working application but it was not really ready for the real world because it had one major flaw: the URL of the AMF endpoint was hardcoded in the client in such a way that it was impossible to change after compilation and very hard to handle several environments (dev, test, prod). The solution to that problem is to integrate dependency injection into the mix.

Now there are a lot of such frameworks for Flex/ActionScript applications, including Parsley, Swiz, Cairngorm, etc. But I’ve never been a big fan of those big MVC frameworks that impose their own interpretation of the MVC pattern and completely limit the initial capabilities of Flex itself. For me, the Flex framework itself is clean enough that you don’t need all that overhead and it’s better to use a non-intrusive framework like Spring ActionScript. So that’s what we are going to do.

JVM Web Framework Survey, First Results

Yesterday at Devoxx, Matt Raible did a very interesting talk on comparing JVM web frameworks. On this occasion he had the incredible courage of voicing his opinion on each of the most well-known frameworks, rating them in a matrix and the craziest part: showing this matrix to everyone.

Immediately after his talk, Twitter was on fire with advocates of each of those frameworks complaining about how those ratings were unfair and biased. I mean of course, I can hardly talk about 3 or 4 of these frameworks with the same level of confidence, but the guy has enough experience to have played with at least 13 of them, and it’s perfectly normal to expect him to have up-to-date and accurate feedback about all of them.

Anyway, his talk was highly entertaining but in the end it inspired me 2 reflections:

  1. His list of 20 criteria is excellent and covers pretty much everything, except maybe for “graphics design integration” which I think is very important and some frameworks make it much easier than other, like Flex with Flash Catalyst for example. So even if you don’t agree with the ratings, you can still reuse his methodology, build 13 proof of concepts and rate them yourself.
  2. Rather than complaining, let’s do a survey.

So right after the talk, I built a small Google docs survey, and to this date I got 26 answers, which I think is far less than the number of complaints, but is already a good start. Here are the first results based on those 26 responses. As you can see, there was a small issue with Google Docs who didn’t correctly save the results for the last criterion, degree of risk. So the rankings are based on only 19 criteria so far. But if you still want to voice your opinion, you can still do it. I will update the results from time to time and this time, the 20th criterion should be considered.

Thanks again Matt Raible for this very inspiring talk, and thanks to all the Devoxx team for yet another memorable edition.

By the way, as far as I’m concerned, I’m pretty happy with Grails and Flex at the moment, but after the amazing demos I saw at Devoxx, I will probably have a deeper look at Vaadin very soon. And please Jetbrains, we need more visual designers (Flex? Vaadin?).