Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bandersnatch verify --delete occurs error #1488

Open
89ao opened this issue Jul 6, 2023 · 2 comments
Open

bandersnatch verify --delete occurs error #1488

89ao opened this issue Jul 6, 2023 · 2 comments
Assignees
@cooperlees
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@89ao
Copy link
Contributor

89ao commented Jul 6, 2023 •
edited by cooperlees
Loading

sh-4.2# bandersnatch --version
bandersnatch 6.3.0

sh-4.2# bandersnatch verify --delete --json-update
2023-07-06 16:13:58,112 bandersnatch.verify: INFO Starting verify for /repo with 10 workers
2023-07-06 16:13:59,897 bandersnatch.verify: INFO Parsing 101903762
2023-07-06 16:13:59,897 bandersnatch.master: INFO Fetching https://pypi.org/pypi/101903762/json
2023-07-06 16:14:00,220 bandersnatch: INFO Initialized release plugin blocklist_release, filtering [<Requirement('tensorflow-io-nightly')>, <Requirement('tf-nightly-cpu')>, <Requirement('tf-nightly')>, <Requirement('appium')>, <Requirement('pyagrum-nightly')>, <Requirement('tf-nightly-gpu')>, <Requirement('uselesscapitalquiz')>]
Traceback (most recent call last):
  File "/usr/local/bin/bandersnatch", line 8, in <module>
    sys.exit(main())
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch/main.py", line 226, in main
    return asyncio.run(async_main(args, config))
  File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch/main.py", line 162, in async_main
    return await bandersnatch.verify.metadata_verify(config, args)
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch/verify.py", line 269, in metadata_verify
    await verify_producer(
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch/verify.py", line 227, in verify_producer
    await asyncio.gather(
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch/verify.py", line 217, in consume
    await verify(
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch/verify.py", line 160, in verify
    plugin.filter(pkg)
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch_filter_plugins/blocklist_name.py", line 146, in filter
    version = metadata["version"]
KeyError: 'version'
@cooperlees
Copy link
Contributor

Interesting, must be a package with no versions. Can you capture the --debug output and attach please and I'll see what the package metadata looks like and make the code handle this metadata state.

Mainly I just want to see the JSON for the package it's failing on to see how to handle it correctly. Should be an easy fix.

@cooperlees cooperlees added bug Something isn't working help wanted Extra attention is needed labels Jul 6, 2023
@cooperlees cooperlees self-assigned this Jul 6, 2023
@89ao
Copy link
Contributor Author

89ao commented Jul 6, 2023

the debug log is below:

sh-4.2# bandersnatch --debug verify --delete
2023-07-06 18:57:04,396 bandersnatch.verify: INFO Starting verify for /repo with 10 workers
2023-07-06 18:57:14,176 bandersnatch.verify: INFO Parsing 101903762
2023-07-06 18:57:14,213 bandersnatch: INFO Initialized release plugin blocklist_release, filtering [<Requirement('tf-nightly-gpu')>, <Requirement('tf-nightly-cpu')>, <Requirement('pyagrum-nightly')>, <Requirement('uselesscapitalquiz')>, <Requirement('tensorflow-io-nightly')>, <Requirement('tf-nightly')>, <Requirement('appium')>]
Traceback (most recent call last):
  File "/usr/local/bin/bandersnatch", line 8, in <module>
    sys.exit(main())
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch/main.py", line 226, in main
    return asyncio.run(async_main(args, config))
  File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch/main.py", line 162, in async_main
    return await bandersnatch.verify.metadata_verify(config, args)
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch/verify.py", line 269, in metadata_verify
    await verify_producer(
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch/verify.py", line 227, in verify_producer
    await asyncio.gather(
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch/verify.py", line 217, in consume
    await verify(
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch/verify.py", line 160, in verify
    plugin.filter(pkg)
  File "/data/bandersnatch/lib/python3.9/site-packages/bandersnatch_filter_plugins/blocklist_name.py", line 146, in filter
    version = metadata["version"]
KeyError: 'version'

sh-4.2# cat /repo/web/simple/101903762/index.html
<!DOCTYPE html>
<html>
  <head>
    <meta name="pypi:repository-version" content="1.0">
    <title>Links for 101903762</title>
  </head>
  <body>
    <h1>Links for 101903762</h1>
    <a href="../../packages/0a/29/8ee8281a7d9f43c544e5bc0b2433449a40d838dca1a444385d54422aa2a0/101903762-0.0.1-py3-none-any.whl#sha256=9ac8b5f28832285c63239412a522d8543d380829323a3a02a66df3a7a3cff346">101903762-0.0.1-py3-none-any.whl</a><br/>
    <a href="../../packages/e1/f4/269740bd163001bcab48975f49b3a0eec70a6919ffdfaea28f74ddb9d7d3/101903762-0.0.1.tar.gz#sha256=07c81e62d19da21e2fd970dfdceec02ddeaed56a746135bf0cfafc7259c0dc32">101903762-0.0.1.tar.gz</a><br/>
  </body>
</html>
<!--SERIAL 13026825-->

the package "101903762"'s json is like this:

sh-4.2# cat /repo/web/pypi/101903762/json
{
    "info": {
        "author": "Divyanshu",
        "author_email": "<[email protected]>",
        "bugtrack_url": null,
        "classifiers": [
            "Development Status :: 1 - Planning",
            "Intended Audience :: Developers",
            "Operating System :: MacOS :: MacOS X",
            "Operating System :: Microsoft :: Windows",
            "Operating System :: Unix",
            "Programming Language :: Python :: 3"
        ],
        "description": "",
        "description_content_type": "",
        "docs_url": null,
        "download_url": "",
        "downloads": {
            "last_day": -1,
            "last_month": -1,
            "last_week": -1
        },
        "home_page": "",
        "keywords": "python,video,stream,video stream,camera stream,sockets",
        "license": "",
        "maintainer": "",
        "maintainer_email": "",
        "name": "101903762",
        "package_url": "https://pypi.org/project/101903762/",
        "platform": "",
        "project_url": "https://pypi.org/project/101903762/",
        "project_urls": null,
        "release_url": "https://pypi.org/project/101903762/0.0.1/",
        "requires_dist": [
            "pandas",
            "numpy"
        ],
        "requires_python": "",
        "summary": "Calculating topsis Score",
        "version": "0.0.1",
        "yanked": false,
        "yanked_reason": null
    },
    "last_serial": 13026825,
    "releases": {
        "0.0.1": [
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "0a298ee8281a7d9f43c544e5bc0b2433449a40d838dca1a444385d54422aa2a0",
                    "md5": "97645b3daf7a93e6b436848780577e49",
                    "sha256": "9ac8b5f28832285c63239412a522d8543d380829323a3a02a66df3a7a3cff346"
                },
                "downloads": -1,
                "filename": "101903762-0.0.1-py3-none-any.whl",
                "has_sig": false,
                "md5_digest": "97645b3daf7a93e6b436848780577e49",
                "packagetype": "bdist_wheel",
                "python_version": "py3",
                "requires_python": null,
                "size": 2272,
                "upload_time": "2022-02-27T18:38:06",
                "upload_time_iso_8601": "2022-02-27T18:38:06.139267Z",
                "url": "https://files.pythonhosted.org/packages/0a/29/8ee8281a7d9f43c544e5bc0b2433449a40d838dca1a444385d54422aa2a0/101903762-0.0.1-py3-none-any.whl",
                "yanked": false,
                "yanked_reason": null
            },
            {
                "comment_text": "",
                "digests": {
                    "blake2b_256": "e1f4269740bd163001bcab48975f49b3a0eec70a6919ffdfaea28f74ddb9d7d3",
                    "md5": "162d6e4f5ad0df4546450e94d39a569a",
                    "sha256": "07c81e62d19da21e2fd970dfdceec02ddeaed56a746135bf0cfafc7259c0dc32"
                },
                "downloads": -1,
                "filename": "101903762-0.0.1.tar.gz",
                "has_sig": false,
                "md5_digest": "162d6e4f5ad0df4546450e94d39a569a",
                "packagetype": "sdist",
                "python_version": "source",
                "requires_python": null,
                "size": 2058,
                "upload_time": "2022-02-27T18:38:08",
                "upload_time_iso_8601": "2022-02-27T18:38:08.263032Z",
                "url": "https://files.pythonhosted.org/packages/e1/f4/269740bd163001bcab48975f49b3a0eec70a6919ffdfaea28f74ddb9d7d3/101903762-0.0.1.tar.gz",
                "yanked": false,
                "yanked_reason": null
            }
        ]
    },
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "0a298ee8281a7d9f43c544e5bc0b2433449a40d838dca1a444385d54422aa2a0",
                "md5": "97645b3daf7a93e6b436848780577e49",
                "sha256": "9ac8b5f28832285c63239412a522d8543d380829323a3a02a66df3a7a3cff346"
            },
            "downloads": -1,
            "filename": "101903762-0.0.1-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "97645b3daf7a93e6b436848780577e49",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 2272,
            "upload_time": "2022-02-27T18:38:06",
            "upload_time_iso_8601": "2022-02-27T18:38:06.139267Z",
            "url": "https://files.pythonhosted.org/packages/0a/29/8ee8281a7d9f43c544e5bc0b2433449a40d838dca1a444385d54422aa2a0/101903762-0.0.1-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "e1f4269740bd163001bcab48975f49b3a0eec70a6919ffdfaea28f74ddb9d7d3",
                "md5": "162d6e4f5ad0df4546450e94d39a569a",
                "sha256": "07c81e62d19da21e2fd970dfdceec02ddeaed56a746135bf0cfafc7259c0dc32"
            },
            "downloads": -1,
            "filename": "101903762-0.0.1.tar.gz",
            "has_sig": false,
            "md5_digest": "162d6e4f5ad0df4546450e94d39a569a",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 2058,
            "upload_time": "2022-02-27T18:38:08",
            "upload_time_iso_8601": "2022-02-27T18:38:08.263032Z",
            "url": "https://files.pythonhosted.org/packages/e1/f4/269740bd163001bcab48975f49b3a0eec70a6919ffdfaea28f74ddb9d7d3/101903762-0.0.1.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "vulnerabilities": []
}

bammab pushed a commit to bammab/bandersnatch that referenced this issue Mar 23, 2024
Fix filtering of verify
a65252d 
Fixes pypa#1488
@bammab bammab mentioned this issue Mar 23, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cooperlees cooperlees

Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@cooperlees @89ao