Show "removed" samples from the sample plot / Give some indication of why

[fedarko]

The solution with #91 provides us with a way to ensure that, e.g., only samples with a numerical metadata value are present in the scatterplot. However, it'd be a good idea to also inform the users of how many samples were excluded due to invalid-ish values.

A really cool example of this is this Vega example, where nulls are represented as gray points along the axes. (Also see a simpler version of the example for Vega-Lite.) However, we don't necessarily need fancy solutions like these for our purposes; I'd be fine having just a little text box or something next to the sample plot that contains the number of "dropped" samples. This could be as simple as the "type": "text" mark on the Vega example above—just count the number of invalid values and say something like {} samples with missing values.

In a sense, this functionality would work double duty—it'd inform the user both about which samples don't have the features contained in the log ratio and about which samples don't have valid metadata for the current x-axis.

[fedarko]

Also, certain color fields can result in samples being dropped. IMO this is a very important consideration.

Right now all of the samples that are passed to the visualization are included in the "export data" output (EXCEPT for samples with an invalid log ratio, i.e. due to missing certain features in the log ratio). We should either make this consistent—and include samples with an invalid log ratio alongside samples with invalid metadata fields—OR we should only include "currently drawn" samples, I guess? In any case, this should be super explicit.

[fedarko]

It'd be cool to show either a tally or a percentage of samples dropped (latter idea c/o Lisa).

[fedarko]

plan of how code for displaying this would work:

• There should be four divs below the sample plot (i.e. just below the boxplot toggle).
  • The first one of these should be updated every time the feature selection(s) change, to let the user know how many samples are not shown in the plot due to having a NaN balance (i.e. 0 in either/both of the numerator/denominator).
    • There's already code in Qurro that computes the number of dropped samples, so this is super feasible.
  • The second one of these should be updated every time the x-axis scale type changes, or every time the x-axis field changes. This should go through all samples to figure out how many samples will not be shown in the plot due to not being treatable as a number (I think we can just use vega.toNumber), and just let the user know how many samples are affected by this.
  • The third one is basically the same as the x-axis one, but for the color field stuff. We should be able to abstract out almost all of the code for this. (And in the future, if we add more encodings, it should be simple to extend this.)

We should store lists of sample IDs that have been excluded by any of these three metrics, so that we can say in the fourth div how many samples are currently shown in the sample plot at all (similar to Emperor's top-left-corner box). We can get the total number of dropped samples by computing the union of the three "excluded samples" lists, which shouldn't take that much time.

The number of shown samples is just # samples - # dropped samples, then.

Also, if any of the divs saying why a sample has been dropped correspond to 0 samples being dropped (e.g. all of the x-axis fields of all samples are quantitative), then hide those divs I guess.

[fedarko]

I've spent some time looking into if it'd be possible to get Vega/Vega-Lite to do some of this work for us—perhaps with the "valid" / "missing" aggregate transforms. But I haven't been able to find much documentation on how those work.

However, I think we can do this by patching the underlying Vega spec. Advantages of this:

1. Minimizes the amount of code I have to write
2. Since we're making Vega do the work for us, less chance of things getting outdated when the Vega* versions used are updated
3. Lets us embed the number of missing samples into the plot directly

Disadvantages:

1. This might cause a lack of granularity? -> actually nah I think we can just define multiple data sources that filter the samples in different ways so we can include multiple counts, like I'd planned
2. Might have a slightly heavier memory footprint due to having to potentially store multiple samples (if we define multiple data fields) but idk
3. Inherently less flexible than doing it ourselves in JS/HTML (e.g. just displaying a percentage in a Vega text mark is going to take me at least 30 minutes to figure out)
4. We'll need to keep patching the Vega spec whenever we call vegaEmbed, which is doable but could get really annoying
5. I have no idea how this will work with boxplots

Basically, the way we could do this is similar to how the nullXY data source is created and used in the Vega example here. We can define a Vega dataset that takes its data from the main dataset, but instead of filtering out data points that are null / NaN on either the x- / y- axis / (/color?) fields we can filter to just data points that are null or NaN on these fields.

This will necessitate some work in order to ensure that—

• every time a scale type is changed to categorical, the NaN checks are removed (I think?)
• every time a scale type is changed to quantitative, the NaN checks are added back in
• every time a scale field is changed, the fields being checked are updated
  Although I think we can just add some code (probs another function) into makeSamplePlot() that handles all of this adequately. The code footprint of this solution probably wouldn't be thaaaat heavy.

Here's a Vega spec that shows something like this in action on the sleep apnea dataset.

I'm not super sure about whether I prefer this or the manual (i.e. using custom HTML and JS) solution. Will think about it.

If we'll go with the Vega solution, make sure to attribute the aforementioned scatter plot null values example in the code.

[fedarko]

Yeah, I think it's going to be easiest to do this manually (without messing around with the generated Vega spec).

Added getValidSamples(), which should be a solid way to figure out omitted samples for a given field. TODOs are:

• unit-test getValidSamples()
• integrate getValidSamples() with makeSamplePlot() (updating the x-axis and color field divs).
• Use all dropped-sample information to compute a total # of samples shown (calling updateMainSampleShownDiv() accordingly).
• end-to-end tests that this works in practice (when Explicitly wait for Vega plots to be created before enabling various interactive things #85 is addressed).
• account for weird crap like "Infinity" or "-Infinity" in quantitative scales, or "" in nominal scales. Either filter these out in makeSamplePlot() (saving a log of their filtering, so they can be added back in when the scale is changed) or just accept the weirdness and update the documentation (this esp. might be more feasible for the infinity problem).
  • I think handling Properly handle missing values in sample/feature metadata #101 (so that all metadata vals are parsed as strings, and ""s are converted to Nones in python (i.e. null in JS)) is the best course of action before this. Then we can add tests that verify that this works in python, and that the sample dropping stats in JS are also handled properly.
• Add tests for aforementioned weird stuff. This will probably tie in with Properly handle missing values in sample/feature metadata #101. We want to make sure all of our decisions in RRVDisplay.getInvalidSampleIDs() match up perfectly with Vega-Lite, and I'm not confident that's the case yet. Need to test that all of these assumptions hold.
  • one alarming thing is that nulls are apparently only filtered by v-lite when the scale is quantitative -- they're fair game in nominal scales. Need to account for this one way or another; should be doable to add a filter transform to the sample plot that removes nulls for the x and color encoding fields?, and update it whenever the current encoding is changed.