Designing Web for Big Data

As a web developer and designer the usual challenges I encounter involve things like making sure the page loads fast, providing users with a good layout that directs them to the content in an efficient manner, ensuring that the navigation makes sense, validating their input to prevent user errors and generally making their browsing experience pleasant. All these issues and their solutions have been documented dozen times over by many people. Most of the specifics of the various issues that arise are known and most of the ways to resolve those specific issues are also known. Once in a while however, I find myself dealing with “big data” and then the question becomes, how do I present the user with a lot of data – visual or textual – in a manner that’s not overwhelming, that makes it easy to navigate said data and more importantly, understand and work with it. Or to put it more bluntly, how the heck do I make a table that has hundreds of rows and dozens of columns not a complete pain to deal with?

In this article I will go over some of the specific issues I encountered when building and designing web pages with large amounts of data and the approaches I took to resolve those issues. Not all of my solutions will be appropriate for every type of problem but hopefully this information will come in handy in the future.

Information at a glance

This is the core tenant of user interface design as it relates to data. When a user goes to a page with a lot of data their first priority is the data, and so it should be our priority, as designers as well. It all comes down to displaying as much information as possible, in as clear and uncluttered way as possible. So of course, it doesn’t mean you put a table with tiny font and small rows just to fit more of them in it, because that would be difficult to read. On the other hand, spacing things too much or putting labels that are too verbose, can also be counterproductive. It’s all about finding balance between data density and data clarity.

So how do you find that balance?

The key is to know your users and their needs. What is a user’s primary purpose on this page? What data are they looking for first? What elements, whether it’s the data itself or the data controls (sort, filter, modify, etc.), are the most important to the user and their task on this page?

Once you know the answers to these questions you can start thinking about how to best fulfill the user’s needs. This usually comes down to two things – choosing what to show right away, what to show on demand, and how to show it.

Let’s look at the how first.

Data visualization

Sometimes you might find yourself dealing with a page that doesn’t display just one type of data but rather displays a kind of summary of data from across a number of different systems. In a recent project I worked on, we had such a page. It was basically the front page of the site and it needed to show statistics and small chunks of data from several core systems of the site. Things like number and type of support tickets, number and status of devices, as well as CPU and memory usage for devices. All this information is available on other pages in great detail but we needed to show some of it on the front page, all together.

In a situation like this, it’s important not to just display the data, but also to clearly differentiate it and give each type of data distinct style. This way the user will quickly learn to identify each distinct type of information, read and understand it fast, and then move on to whatever it is they need to do.

The best way to differentiate different types of data is to give them different visual styles. So for example what we did was to present the support ticket information as a pie chart, while the device information is presented as a bar chart. It’s a simple and seemingly trivial distinction, but what it does is engage the user’s visual memory. Now, instead of scanning a bunch of text or tables and trying to remember where the ticket information is the user learns to recognize a pie chart as related to support tickets and bar chart as related to devices. So if they’re only interested in the numbers of support tickets then they’ll be able to glance at that information, absorb it and move on, without wasting valuable time searching for it.

Complex forms

Sometimes however, charts are not really appropriate, for example when dealing with forms. In my work sometimes I have to design forms with dozen or so text fields, several drop down lists and a number of checkboxes or radio buttons. These kinds of forms become unwieldy quickly and are a challenge to deal with. To be honest I still struggle with designing such forms – there just doesn’t seem a good way to present user with all that information without the page becoming cluttered. I did however find some ways to reduce the clutter.

Good use of labels and groupings can go a long way towards decluttering the form. For example you could group some fields under the general label of “address” (city, street, postal code, etc.), others can go under “contact” (email, phone, website), and so on.

Spacing helps too. Obviously you would space out the groups but don’t be afraid of spacing out the fields themselves as well. Forms can tolerate bigger spacing (before it becomes awkward) than other types of data, in part because the form elements themselves have more distinct structure – buttons, fields, lists, they’re all boxes with clear boundaries. The other factor is that usually form traversal is done via the Tab or Enter/Return key and so the distance between the fields is not as relevant. By using bigger spacing in forms you create a cleaner look for each individual field (there are less distracting elements around it), which makes it easier for the user to concentrate on their task.

Finally, in some cases you can hide certain elements of the form until they’re necessary. For example, if your form requires the user to input their phone number you can initially present them with just one field for the primary phone number. Then add a button to allow the user to add more phone fields if needed. The disadvantage of that method is that it might potentially hide elements that the user should see right away. However, as long as you make sure not to hide any required fields and make it clear to the user that there are more elements just behind a button click, it will keep the form clean while giving the user more options if they want them.

Huge tables

Huge tables are probably the hardest thing to design for. There’s just only so much we can do when the table needs to have a dozen or more columns – things are going to be cluttered no matter what. Nonetheless, even if we can’t design an ideal experience, we can design a better experience, and with tables we need to use pretty much every trick described above.

Visualizing data in a table might not seem like a great idea, but not every visualization has to be a chart. Putting an icon instead of a word also falls under the category of visualizing data. So, if your table displays status of devices, maybe instead of “up” and “down” you could show an arrow pointing up or down. Something like CPU usage can of course be displayed as simply a string of text (85%) but if you visualize it as a string of text with a coloured “progress bar” (say red for 85% and green for 40% or less) behind it then the user will “get” that information that much faster.

By visualizing some of the data in a table you might be able to save some space and present more data without overwhelming the user, but if you’re dealing with hundreds of rows, and more importantly, dozen or more columns, visualization alone won’t be enough. At that point you have to either allow the table to exceed page width, which results in horrible horizontal scrolling, or hide some of the columns and give user the ability to show and hide columns. While the second option is somewhat cumbersome and forces the user to fiddle around with table controls, it is I believe preferable to horizontal scrolling.

This is where knowledge of the user comes in really handy. You need to know what information is absolutely necessary to those viewing the table and what information can be hidden by default and only shown if explicitly requested by the user. By providing the user with the tools to see all the information available, but initially only showing the most critical data, you basically let the user control the density vs. clarity of data balance.


In the end, displaying large amounts of data on a web page, always results in some compromise – you either prioritize the amount of data, or its clarity – and it’s always about trying to get that perfect balance. Using visualization techniques and by prioritizing certain data you can go far in terms of achieving high density and clarity. In the end though, if you have the ability, give user the tools to customize their view of the data themselves, because they’re the only who really know what they want.

Be the first to comment on this post

Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>