How Not To Fooled By Sparkling Visualizations

Not all visualizations are created equal.

Some are there to aid you. Some are there to confuse you.

Don't believe everything you see!

I'm going to give you some pointers on how to decide whether you can believe it or not.

A couple of years ago, I was working in a Fintech co-working space. As usual, I was teaching Data.

During a break, I opened the Financial Quarter Report, an internal quarter journal on the situation of FinTech in Europe. One of these pages attempted to make a point with a chart.

The sum of the percentages on some of these bars amounted to 101%!!

Sparkling and Wrong Viz.

I was so shocked I almost spit out my coffee.

After the break, I used that chart as an exercise for my students to identify all the flaws in this visualization. The whole list took about 15/20 minutes with my students talking one after the other.

Would you believe anything said with ostensively wrong visual support?

Would you make a business transaction with someone using such wrong visual support?

“Being trustworthy requires: Doing the right thing. And doing things right.” - Don Peppers

Unfortunately, since then I have acquired an entire deck of wrong visualisations I use for my students to practice on.

This is not a rare occurrence.

There is a Clear Advantage to Getting the Visualisations Wrong.

This is undoubtedly an excellent way to ascertain whether or not people are paying sufficient attention. I would strongly advise against doing this with your clients or stakeholders.

If you are in the audience and you believe a bad visualisation, you will:

1. Believe a point that has not been proven or is wrong.

   
   

2. Create a strategy for your company based on false assumptions.

   
   

3. Waste a lot of time and money on said strategy.

   
   

4. Have to do the whole work again to check the truth by numbers.

So, here's the 4-question/answer list on what to ask before looking at the bright upward line or the colourful bars.

Here's what to look at.

Let's help Chris, our friendly neighbourhood course creator. A friend of hers, Cat, is trying to convince her to co-found a pet food shop. This friend is using several charts and she wants to know whether to believe this chart or not.

Question 1: What is the Sample Size Used?

The sample size is the number of people, items, or pets considered when drafting the numbers for a visualisation.

Cat is determined to sell fish. Everyone's pet loves it!

Cat proves his point with this one chart:

Favourite Pet's Food

Chris stops and asks, "How many pet owners did you interview? What kind of pets did they have?"

Cat confidently states that all the cat owners in the neighbourhood answered his form.

They all said their pets loved fish!

Chris stops and thinks. It is possible that nowadays there are neighbourhoods without dog owners, but it is unlikely.

Is this chart usable?

A Good Answer.

The form should have been answered by owners representing all kinds of pets in the area. To cover all the possibilities.

Question 2: What is the Size of the Sample Used.

It is as important to consider the diversity of the source as it is to consider the size of the source. If it's too small, we won't be able to generalise the results.

Chris then asks, in response to the previous answer: "And how many cat owners are there in the neighbourhood?"

Cat answers: "Three!" he states emphatically.

A Good Answer.

For statistical purposes, the answer should be at least 6, but 30/50 would be more accurate. Anything over 10,000 is amazing.

There are only three pet owners in this neighbourhood? Then three is the answer you're looking for.

But this is also the wrong neighbourhood to set up a pet food shop.

Question 3: Where does Your Data Come From?

The source is a clear indication of the quality and truthfulness of the answers. The source is crucial.

Chris is relentless in pursuing the topic at hand. She asks the question again: "And how did you find these pet owners?"

Cat proudly states that he met a number of friendly individuals at the neighbourhood course of fitness, who were happy to answer his questions.

A Good Answer.

There are plenty of official websites with data on social topics.

You need an official source!

Note: if you are making a survey on yoga practitioners, maybe foryour studio purposes, then Cat's answer might be the official source.

Question 4: Can the visualization to convey the point being made?

The topic and details of the point being made must match the topic of the visualisation.

Be careful with this one. This is the one thing more people get wrong. Sometimes, data is absent or incorrect, and "generally adjacent" data is used instead. However, similar data does not prove the specific point.

If the intention is to convey that fish should be sold as the main pet food shop item, then a survey of residents to determine their favourite food is not the appropriate method. It must at least have to do with their pets.

A Good Answer.

Cat's survey asks about the pet's favourite food. He should have asked about a list of any food their pets eat, but he's on the right track.

It would have been too sad if Cat wouldn't have nailed at least one...

Any More Questions?

You can always add more questions to get to the heart of the situation. Sometimes you might find people reluctant to give answers straight away, so a bit more prompting would help.

After that, you can decide whether to use what you learn from this or that visualisation.

Just ask and trust yourself!

You can do it.