Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
Chris Jewell
covid19uk
Commits
d6dedac1
Commit
d6dedac1
authored
Mar 24, 2021
by
Chris Jewell
Browse files
Added dynamic re-try for data download
parent
d8069a6a
Changes
2
Hide whitespace changes
Inline
Side-by-side
covid/data/case_data.py
View file @
d6dedac1
"""Loads COVID-19 case data"""
import
sys
from
warnings
import
warn
import
requests
import
json
...
...
@@ -45,10 +46,23 @@ class CasesData:
"""
Placeholder, in case we wish to interface with an API.
"""
response
=
requests
.
get
(
url
)
content
=
json
.
loads
(
response
.
content
)
df
=
pd
.
read_json
(
json
.
dumps
(
content
[
"body"
]))
return
df
max_tries
=
5
secs
=
5
for
i
in
range
(
max_tries
):
try
:
print
(
"Attempting to download..."
,
end
=
""
,
flush
=
True
)
response
=
requests
.
get
(
url
)
content
=
json
.
loads
(
response
.
content
)
df
=
pd
.
read_json
(
json
.
dumps
(
content
[
"body"
]))
print
(
"Success"
,
flush
=
True
)
return
df
except
ConnectionResetError
:
print
(
"Failed"
,
flush
=
True
)
sys
.
sleep
(
secs
*
2
**
i
)
raise
ConnectionError
(
f
"Data download timed out after
{
max_tries
}
attempts"
)
def
getCSV
(
file
):
"""
...
...
pyproject.toml
View file @
d6dedac1
...
...
@@ -38,10 +38,13 @@ black = "^20.8b1"
pytest
=
"^6.2.1"
jedi
=
"^0.17.2"
[tool.poetry-dynamic-versioning]
enable
=
true
[tool.black]
line-length
=
80
include
=
'\.pyi?$'
[build-system]
requires
=
["poetry-core>=1.0.0"]
requires
=
["poetry-core>
=
1.0
.
0
"
, "
poetry-dynamic-versioning
"
]
build-backend
=
"poetry.core.masonry.api"
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment