Skip to content

Fix issue3296: prevent writing when running tests in CDMS #3297

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 21, 2025

Conversation

keflavich
Copy link
Contributor

Simple fix for #3296. We do want the remote tests grabbing stuff from the remote server, but you're right, we don't want it modifying data files in the repository.

Copy link
Member

@bsipocz bsipocz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, as this is a new kwarg, it will need a changelog entry.
And maybe consider being a bit more verbose with naming the kwarg, but it's not a blocker.

use_cached : bool, optional
If True, use the cached file if it exists. If False, download the
file from the CDMS server and save it to the cache (if ``write`` is set).
write : bool, optional
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A more verbose arg would be nice, maybe call this write_cached or refresh_cashed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll be more verbose, but I think a different keyword is warranted... write_new_cache, probably.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Anything more that just write is an improvement, and write_new_cache is quite good. Though maybe it will need a switch in logic, e.g. if it's set then ignore whatever is in use_cache rather than the other way around.

Copy link
Member

@bsipocz bsipocz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird that the CI wasn't kicking in, I'm fairly certain one of the lines is too long. I also now see uncatched warnings, please address those and then we can merge this.

@@ -317,6 +327,8 @@ def get_species_table(self, *, catfile='partfunc.cat', use_cached=True,
"""

if use_cached:
if write_new_species_cache:
warnings.warn("use_cached and write_new_species_cache are both set to True; write_new_species_cache will be ignored. If you meant to update the cache, set use_cached to False.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now issues a couple of warnings in the tests that we need to handle.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uhoh.... that shouldn't be happening.

…urity' to prevent overwriting, but that's not needed
Copy link

codecov bot commented Apr 21, 2025

Codecov Report

Attention: Patch coverage is 25.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 69.47%. Comparing base (4ed16e4) to head (213a0cd).
Report is 14 commits behind head on main.

Files with missing lines Patch % Lines
astroquery/linelists/cdms/core.py 25.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3297      +/-   ##
==========================================
+ Coverage   69.36%   69.47%   +0.10%     
==========================================
  Files         232      232              
  Lines       19692    19707      +15     
==========================================
+ Hits        13659    13691      +32     
+ Misses       6033     6016      -17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

linelists.cdms
^^^^^^^^^^^^^^

- Add a keyword to control writing of new species cache files. This is needed to prevent tests from overwriting those files. [#3300]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're living in the future :) (But I'll fix it at release time :) )

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh shoot, I had fixed it and just didn't push yet... I got some failing tests still to fix

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did run the tests prior merging and it was all clear (besides a ConnectionError, but I double checked and that one was present on main, too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I needed to update my local table....

Anyway, I have a fix to push because this last test took too long to run for my liking

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say don't stress on the retrieval one as we don't run it often and it's only 2s on my local. But if you would fix the really long running (400+s )and then failing one, the test_regression_allcats, that would be great.

@bsipocz bsipocz merged commit 7ac08b8 into astropy:main Apr 21, 2025
10 of 12 checks passed
@bsipocz
Copy link
Member

bsipocz commented Apr 21, 2025

Thanks!

@bsipocz bsipocz linked an issue Apr 21, 2025 that may be closed by this pull request
@keflavich
Copy link
Contributor Author

There might be a related failing test - I'm not sure yet. I got this error, but it coincided with a server error:

___________________________________________________________________________________________________________ test_regression_allcats ____________________________________________________________________________________________________________

    @pytest.mark.bigdata
    @pytest.mark.remote_data
    def test_regression_allcats():
        """
        Expensive test - try all the molecules
        """
        species_table = CDMS.get_species_table(write_new_species_cache=False)
        for row in species_table:
            tag = f"{row['tag']:06d}"
>           result = CDMS.get_molecule(tag)

astroquery/linelists/cdms/tests/test_cdms_remote.py:133:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
astroquery/linelists/cdms/core.py:399: in get_molecule
    result = self._parse_cat(response)
astroquery/linelists/cdms/core.py:469: in _parse_cat
    intcol = np.array(list(map(parse_letternumber, result[key])),
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

st = '13-'

    def parse_letternumber(st):
        """
        Parse CDMS's two-letter QNs

        From the CDMS docs:
        "Exactly two characters are available for each quantum number. Therefore, half
        integer quanta are rounded up ! In addition, capital letters are used to
        indicate quantum numbers larger than 99. E. g. A0 is 100, Z9 is 359. Small
        types are used to signal corresponding negative quantum numbers."
        """
        asc = string.ascii_lowercase
        ASC = string.ascii_uppercase
        newst = ''.join(['-' + str(asc.index(x)+10) if x in asc else
                         str(ASC.index(x)+10) if x in ASC else
                         x for x in st])
>       return int(newst)
E       ValueError: invalid literal for int() with base 10: '13-'

astroquery/linelists/cdms/core.py:497: ValueError

I'm re-running.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: cdms test_retrieve_species_table overwrites test file
2 participants