hello my name is Devin Lange I'm a PhD student at the University of Utah and
the current member of the visualization Design Lab in this video I'm going to be
talking about Ferret our recent work to help review tabular data sets for manipulation and
in particular this video is going to focus on a specific case study of the the tool
so to begin let me give a quick overview of the interface on the left hand side we have
a number of analyzes each one of these analyzes is designed to help highlight patterns that that
indicate that data may have been manipulated in the top middle portion of the screen we
include a description that describes the the visual the visualizations as well as what to
look for and caveats to keep in mind since I'm going through the demo I'm going to hide
these descriptions so we have more space for our main view which is the the tabular visualization
a quick overview for the data that we're looking at in this case study this is associated with a
paper that has been retracted in particular experiment number three and the way this
experiment is described in the paper is that the researchers worked with a insurance company
and uh recorded asked people to record their odometer reading of their car how
many miles that car had driven then after some period had elapsed paper didn't go
into details for exactly what what period of time after this period of time elapsed
the the uh owners of the vehicles were asked to report their odometer reading again and
since each row here is a actual insurance policy there can be up to four cars on it but most of the
policies only have a single car listed for each car there is a previous column the odometer at the
beginning of this period and an update column the the odometer reading after this period has elapsed
so with that let's talk about why some of these cells are highlighted in blue and some are white
if you hover over a cell it explains all of the formatting things like the font and font size
and these things are what Drive the um the the the formatting that you see on the screen
so we see that there are 20 000 cells that have a font size of 1200 and have a font of Cambria
however most of the cells are actually in calibri font so already we can see in this first column
uh there's this sort of strange mix of these two different fonts throughout the data
and we can switch to an overview mode that lets us view many many columns at once
and if we scroll through our data set we can see that that this pattern continues throughout the
the 13 000 rows or so where there's this mix of these two fonts in this column this column
the odometer reading One update is all Cambria and then the rest appear to be
calibri so let me quickly go back to the top of this list and I'm going
to now sort by the values within this column and here once we do this new patterns emerge
so instead of this this random mixing of these two fonts the first thing we can notice is that
uh every single value of zero within this region is going to be in this calibri font right there
are there so there's uh our first difference between the values in calibri and Cambria
similarly if we look at the next range of numbers close to a thousand right this
period between uh 600 and 500 and a thousand um is not entirely in this blue font but it is
mostly this this blue font now if we continue down we have more of a a mixing of fonts so to speak
but you may notice if you look carefully that there are some regions
that have larger consecutive chunks of white fonts so for instance here and here are fairly
large chunks of this white font but we don't see the same thing with the blue fonts so if we look
at the actual values in those regions what we see is that they are these rounded numbers so 37 000
38 000 . um and forty thousand and interestingly the the rounder the number the larger list chunk
is so between these these numbers 40 000 is the the biggest so if you're looking at data
that has been self-reported like odometer readings reported by owners of the vehicle It's Not Unusual
for there to be this sort of rounding effect instead of looking at the actual reading they
just estimate and go with the the closest number that's reasonable so that by itself is Not Unusual
um but if we look at the data we do not see that same rounding effect in the blue font
so again here there's a a difference in the data within a single column between these two
okay moving on
um to The Other Extreme of the data
if we move to the largest values within this column we see a new pattern
and I can click and drag to highlight many rows at once and now we can see this interesting pattern
where values alternate between this blue font and this white font calibrian Cambria
and furthermore we see pairings of numbers so the largest two numbers in the data set 982 and 983
000 are within one thousand miles of each other and then the next two closest also
share this pattern and if you go one by one you see this for every single pattern here
where every single excuse me pair here where the two values are within 1000 miles of each other
and in fact if you look at the the second third fourth cars on the policy for insurance policies
that have more than one car you see the same pattern right these two odometers are within
a thousand miles these two are within a thousand miles um and so on so there's
a bit of a strange uh uh pattern of of matching values at this tail end of the distribution
okay so I'm going to move on to our next analysis here so I will skip the
value distribution and go to the duplicate numbers
this uh and I'm going to just make this a little bit larger here to make it easier to see
this analysis is showing values that are duplicated within a single column
so for odometer reading one previous we can see that
the most duplicated numbers are again these very round numbers right sixty thousand
was duplicated 10 times in other words there were excuse me 30 times there were 30 cars
within this data set that reported an odometer of sixty thousand miles
uh on the other hand the second column the odometer reading
after some time has passed the update column you do not see these round numbers and the the
amount of duplicates are much higher lower three instead of 30 for instance
for the second car the values of zero actually in this case correspond to not having a second
car but we can ignore those values actually in any column and with those ignored we see the
same pattern for the second car the previous has round numbers many duplicates the updated
does not have round numbers no duplicates so again there seems to be some difference
um between these columns for the previous column and the update column now we see the
same type of rounding effect in the replicate analysis and the duplicate digit analysis
but I'm going to skip to the the trailing digits which shows this perhaps the most clearly
right this uh chart is looking at the the relative frequency of the last digits of numbers so
again for the First Column you can see that 20 percent roughly of the numbers end with a zero
and then the rest are uniform compared to the update where all of the values are uniform and
again we see the same pattern in the second third fourth cars so the last thing I'm going
to switch to is this last analysis listed here for checking your domain expectations
um and this provides a collection of different visualizations to help you analyze your data
so the first thing I'm going to do is I'm going to pull in our odometer reading one for the previous
and update this is what we've been looking at and this is going to plot a scatter plot
so if I reduce the opacity here and let me reset this actually
great so now that we have this I want to just point out a couple quick things the
bottom axis is the previous and the top X the y-axis is the the update
so to read this if you look at you know a point right here that means the car started with 200
000 miles and then didn't drive it all at the end of this time period it was still at 200 000 miles
so in other words everything on this sort of uh line at the bottom of this boundary are all cars
that drove zero miles or very close to zero miles so it's good we don't see anything below this
um because those would all indicate negative milestone on the other hand if we look at
this Top Line right we can look at for instance this point this is a car that
started with odometer reading of zero drove 50 000 miles and then at the endless idea drove 50.
yeah it had 50 000 miles greater on its odometer so everything on this upper line are cars that
drove 50 000 miles so given the the number of cars roughly fifteen thousand total um and
the the density there it is a bit strange to see this very abrupt cutoff with so many cars driving
close to 50 000 miles but no cars driving you know 51 000 miles or or any any any value greater
okay so that concludes the the abbreviated version of this case study
if you're interested in learning more about the tool check out the paper thank you very much yeah