fix: keep FlyBase gene nan when reading 10x mtx

[joncarse]

• Closes ValueError: cannot convert float NaN to integer #1259
• Tests included
• Release notes not necessary because:

FlyBase has a real gene named nan. read_10x_mtx reads genes.tsv / features.tsv with pd.read_csv, and pandas treats nan as missing by default, so that symbol disappeared. That showed up as calculate_qc_metrics blowing up after var_names.str.startswith(...) (#1259, also #1708), or as a silently wrong identifier.

This keeps lowercase nan as a gene name and still treats the rest of pandas’ NA tokens (NaN, NA, and so on) as missing. The default token list is copied in the reader so production code does not import pandas._libs; a test compares that copy to pandas so we notice if it drifts.

barcodes.tsv is unchanged.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.93%. Comparing base (cdd3d6f) to head (a98108d).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4329   +/-   ##
=======================================
  Coverage   81.92%   81.93%           
=======================================
  Files         134      134           
  Lines       13027    13030    +3     
=======================================
+ Hits        10673    10676    +3     
  Misses       2354     2354           

Flag	Coverage Δ
hatch-test.low-vers	78.81% <100.00%> (+<0.01%)	⬆️
hatch-test.pre	81.80% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/scanpy/io/_read.py	80.81% <100.00%> (+0.21%)	⬆️

[joncarse]

I doubt "nan" will ever appear as truly meaning a nullish value in any TSV file. If this is a concern we can technically filter out "nan" only for Drosophila analyses. But then again maybe someone will try to splice it into another organism :)

[flying-sheep]

Thanks!

First a rant I can’t keep myself from: I hate CSV/TSV 😭 I’ve been telling people my whole career they should avoid it yet I have little success, and bullshit like this makes it into some industry standard.

For this PR, I like that you protect against drift, but I changed it to be simpler:

pd.read_csv(path, sep="\t", header=None, dtype=str, na_filter=False, quoting=csv.QUOTE_NONE)

and removed the astype("str") which is then redundant.

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:
They assume that the base project is the Git remote upstream and your fork is origin.

1. Update backport branch.

git fetch upstream 1.12.x:1.12.x
# or `git pull upstream 1.12.x` if you are on 1.12.x

2. Create PR branch.

git switch -c auto-backport-of-pr-4329-on-1.12.x 1.12.x

3. Cherry pick the first parent branch of this PR on top of the older branch:

git cherry-pick -x -m1 a41e03fd22b4448c7c85fa8d6c3deda49699caae

4. You will likely have some merge/cherry-pick conflict here, fix them and commit:

git commit -am 'Backport PR #4329: fix: keep FlyBase gene nan when reading 10x mtx'

5. Push to a named branch:

git push --set-upstream origin auto-backport-of-pr-4329-on-1.12.x

6. Create a PR against branch 1.12.x, I would have named this PR:

Backport PR #4329 on branch 1.12.x (fix: keep FlyBase gene nan when reading 10x mtx)

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

[joncarse]

@flying-sheep thanks for the review and corrections, yes we would all be happier without CSV/TSV. Your fix is much simpler, the drift protection was overkill.

History taught me not to leave Mr. Meeseeks unhappy for too long, so I created #4377 as instructed