Python: What happens if script stops while requests.get() is executing?

Clash Royale CLAN TAG#URR8PPP
up vote
14
down vote
favorite
I know that requests.get() provides an HTTP interface so that the programmer can make various requests to a HTTP server.
That tells me that somewhere a port must be opened so that the request can happen.
Taking that into account, what would happen if the script is stopped (say, by a Key Board Interrupt, so the machine that is executing the script remains connected to the internet) before the request is answered/complete?
Would the port/connection remain opened?
Does the port/connection close automatically?
python http python-requests python-requests-html
add a comment |Â
up vote
14
down vote
favorite
I know that requests.get() provides an HTTP interface so that the programmer can make various requests to a HTTP server.
That tells me that somewhere a port must be opened so that the request can happen.
Taking that into account, what would happen if the script is stopped (say, by a Key Board Interrupt, so the machine that is executing the script remains connected to the internet) before the request is answered/complete?
Would the port/connection remain opened?
Does the port/connection close automatically?
python http python-requests python-requests-html
add a comment |Â
up vote
14
down vote
favorite
up vote
14
down vote
favorite
I know that requests.get() provides an HTTP interface so that the programmer can make various requests to a HTTP server.
That tells me that somewhere a port must be opened so that the request can happen.
Taking that into account, what would happen if the script is stopped (say, by a Key Board Interrupt, so the machine that is executing the script remains connected to the internet) before the request is answered/complete?
Would the port/connection remain opened?
Does the port/connection close automatically?
python http python-requests python-requests-html
I know that requests.get() provides an HTTP interface so that the programmer can make various requests to a HTTP server.
That tells me that somewhere a port must be opened so that the request can happen.
Taking that into account, what would happen if the script is stopped (say, by a Key Board Interrupt, so the machine that is executing the script remains connected to the internet) before the request is answered/complete?
Would the port/connection remain opened?
Does the port/connection close automatically?
python http python-requests python-requests-html
asked Aug 13 at 23:12
machetazo
422217
422217
add a comment |Â
add a comment |Â
2 Answers
2
active
oldest
votes
up vote
12
down vote
accepted
The short answer to the question is: requests will close a connection in the case of any exception, including KeyboardInterrupt and SystemExit.
A little digging into the requests source code reveals that requests.get ultimately calls the HTTPAdapter.send method (which is where all the magic happens).
There are two ways in which a request might be made within the send method: chunked or not chunked. Which send we perform depends on the value of the request.body and the Content-Length header:
chunked = not (request.body is None or 'Content-Length' in request.headers)
In the case where the request body is None or the Content-Length is set, requests will make use of the high-level urlopen method of urllib3:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
# ...
)
The finally block of the urllib3.PoolManager.urlopen method has code that handles closing the connection in the case where the try block didn't execute successfully:
clean_exit = False
# ...
try:
# ...
# Everything went great!
clean_exit = True
finally:
if not clean_exit:
# We hit some kind of exception, handled or otherwise. We need
# to throw the connection away unless explicitly told not to.
# Close the connection, set the variable to None, and make sure
# we put the None back in the pool to avoid leaking it.
conn = conn and conn.close()
release_this_conn = True
In the case where the response can be chunked, requests goes a bit lower level and uses the underlying low level connection provided by urllib3. In this case, requests still handles the exception, it does this with a try / except block that starts immediately after grabbing a connection, and finishes with:
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
# ...
except:
# If we hit any problems here, clean up the connection.
# Then, reraise so that we can handle the actual exception.
low_conn.close()
raise
Interestingly the connection may not be closed if there are no errors, depending on how you have configured connection pooling for urllib3. In the case of a successful execution, the connection is put back into the connection pool (though I cannot find a _put_conn call in the requests source for the chunked send, which might be a bug in the chunked work-flow).
1
Thank you very much!
â machetazo
Aug 14 at 14:21
add a comment |Â
up vote
1
down vote
On a much lower level, when a program exits, the OS kernel closes all file descriptors opened by that program. These include network sockets.
Very interesting. Even more interesting is that, as the accepted answer says, the connection may not be closed if there are no errors... would be nice to explore that...
â machetazo
Aug 20 at 19:50
add a comment |Â
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
12
down vote
accepted
The short answer to the question is: requests will close a connection in the case of any exception, including KeyboardInterrupt and SystemExit.
A little digging into the requests source code reveals that requests.get ultimately calls the HTTPAdapter.send method (which is where all the magic happens).
There are two ways in which a request might be made within the send method: chunked or not chunked. Which send we perform depends on the value of the request.body and the Content-Length header:
chunked = not (request.body is None or 'Content-Length' in request.headers)
In the case where the request body is None or the Content-Length is set, requests will make use of the high-level urlopen method of urllib3:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
# ...
)
The finally block of the urllib3.PoolManager.urlopen method has code that handles closing the connection in the case where the try block didn't execute successfully:
clean_exit = False
# ...
try:
# ...
# Everything went great!
clean_exit = True
finally:
if not clean_exit:
# We hit some kind of exception, handled or otherwise. We need
# to throw the connection away unless explicitly told not to.
# Close the connection, set the variable to None, and make sure
# we put the None back in the pool to avoid leaking it.
conn = conn and conn.close()
release_this_conn = True
In the case where the response can be chunked, requests goes a bit lower level and uses the underlying low level connection provided by urllib3. In this case, requests still handles the exception, it does this with a try / except block that starts immediately after grabbing a connection, and finishes with:
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
# ...
except:
# If we hit any problems here, clean up the connection.
# Then, reraise so that we can handle the actual exception.
low_conn.close()
raise
Interestingly the connection may not be closed if there are no errors, depending on how you have configured connection pooling for urllib3. In the case of a successful execution, the connection is put back into the connection pool (though I cannot find a _put_conn call in the requests source for the chunked send, which might be a bug in the chunked work-flow).
1
Thank you very much!
â machetazo
Aug 14 at 14:21
add a comment |Â
up vote
12
down vote
accepted
The short answer to the question is: requests will close a connection in the case of any exception, including KeyboardInterrupt and SystemExit.
A little digging into the requests source code reveals that requests.get ultimately calls the HTTPAdapter.send method (which is where all the magic happens).
There are two ways in which a request might be made within the send method: chunked or not chunked. Which send we perform depends on the value of the request.body and the Content-Length header:
chunked = not (request.body is None or 'Content-Length' in request.headers)
In the case where the request body is None or the Content-Length is set, requests will make use of the high-level urlopen method of urllib3:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
# ...
)
The finally block of the urllib3.PoolManager.urlopen method has code that handles closing the connection in the case where the try block didn't execute successfully:
clean_exit = False
# ...
try:
# ...
# Everything went great!
clean_exit = True
finally:
if not clean_exit:
# We hit some kind of exception, handled or otherwise. We need
# to throw the connection away unless explicitly told not to.
# Close the connection, set the variable to None, and make sure
# we put the None back in the pool to avoid leaking it.
conn = conn and conn.close()
release_this_conn = True
In the case where the response can be chunked, requests goes a bit lower level and uses the underlying low level connection provided by urllib3. In this case, requests still handles the exception, it does this with a try / except block that starts immediately after grabbing a connection, and finishes with:
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
# ...
except:
# If we hit any problems here, clean up the connection.
# Then, reraise so that we can handle the actual exception.
low_conn.close()
raise
Interestingly the connection may not be closed if there are no errors, depending on how you have configured connection pooling for urllib3. In the case of a successful execution, the connection is put back into the connection pool (though I cannot find a _put_conn call in the requests source for the chunked send, which might be a bug in the chunked work-flow).
1
Thank you very much!
â machetazo
Aug 14 at 14:21
add a comment |Â
up vote
12
down vote
accepted
up vote
12
down vote
accepted
The short answer to the question is: requests will close a connection in the case of any exception, including KeyboardInterrupt and SystemExit.
A little digging into the requests source code reveals that requests.get ultimately calls the HTTPAdapter.send method (which is where all the magic happens).
There are two ways in which a request might be made within the send method: chunked or not chunked. Which send we perform depends on the value of the request.body and the Content-Length header:
chunked = not (request.body is None or 'Content-Length' in request.headers)
In the case where the request body is None or the Content-Length is set, requests will make use of the high-level urlopen method of urllib3:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
# ...
)
The finally block of the urllib3.PoolManager.urlopen method has code that handles closing the connection in the case where the try block didn't execute successfully:
clean_exit = False
# ...
try:
# ...
# Everything went great!
clean_exit = True
finally:
if not clean_exit:
# We hit some kind of exception, handled or otherwise. We need
# to throw the connection away unless explicitly told not to.
# Close the connection, set the variable to None, and make sure
# we put the None back in the pool to avoid leaking it.
conn = conn and conn.close()
release_this_conn = True
In the case where the response can be chunked, requests goes a bit lower level and uses the underlying low level connection provided by urllib3. In this case, requests still handles the exception, it does this with a try / except block that starts immediately after grabbing a connection, and finishes with:
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
# ...
except:
# If we hit any problems here, clean up the connection.
# Then, reraise so that we can handle the actual exception.
low_conn.close()
raise
Interestingly the connection may not be closed if there are no errors, depending on how you have configured connection pooling for urllib3. In the case of a successful execution, the connection is put back into the connection pool (though I cannot find a _put_conn call in the requests source for the chunked send, which might be a bug in the chunked work-flow).
The short answer to the question is: requests will close a connection in the case of any exception, including KeyboardInterrupt and SystemExit.
A little digging into the requests source code reveals that requests.get ultimately calls the HTTPAdapter.send method (which is where all the magic happens).
There are two ways in which a request might be made within the send method: chunked or not chunked. Which send we perform depends on the value of the request.body and the Content-Length header:
chunked = not (request.body is None or 'Content-Length' in request.headers)
In the case where the request body is None or the Content-Length is set, requests will make use of the high-level urlopen method of urllib3:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
# ...
)
The finally block of the urllib3.PoolManager.urlopen method has code that handles closing the connection in the case where the try block didn't execute successfully:
clean_exit = False
# ...
try:
# ...
# Everything went great!
clean_exit = True
finally:
if not clean_exit:
# We hit some kind of exception, handled or otherwise. We need
# to throw the connection away unless explicitly told not to.
# Close the connection, set the variable to None, and make sure
# we put the None back in the pool to avoid leaking it.
conn = conn and conn.close()
release_this_conn = True
In the case where the response can be chunked, requests goes a bit lower level and uses the underlying low level connection provided by urllib3. In this case, requests still handles the exception, it does this with a try / except block that starts immediately after grabbing a connection, and finishes with:
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
# ...
except:
# If we hit any problems here, clean up the connection.
# Then, reraise so that we can handle the actual exception.
low_conn.close()
raise
Interestingly the connection may not be closed if there are no errors, depending on how you have configured connection pooling for urllib3. In the case of a successful execution, the connection is put back into the connection pool (though I cannot find a _put_conn call in the requests source for the chunked send, which might be a bug in the chunked work-flow).
edited Aug 14 at 0:45
answered Aug 13 at 23:38
Matthew Story
1,679318
1,679318
1
Thank you very much!
â machetazo
Aug 14 at 14:21
add a comment |Â
1
Thank you very much!
â machetazo
Aug 14 at 14:21
1
1
Thank you very much!
â machetazo
Aug 14 at 14:21
Thank you very much!
â machetazo
Aug 14 at 14:21
add a comment |Â
up vote
1
down vote
On a much lower level, when a program exits, the OS kernel closes all file descriptors opened by that program. These include network sockets.
Very interesting. Even more interesting is that, as the accepted answer says, the connection may not be closed if there are no errors... would be nice to explore that...
â machetazo
Aug 20 at 19:50
add a comment |Â
up vote
1
down vote
On a much lower level, when a program exits, the OS kernel closes all file descriptors opened by that program. These include network sockets.
Very interesting. Even more interesting is that, as the accepted answer says, the connection may not be closed if there are no errors... would be nice to explore that...
â machetazo
Aug 20 at 19:50
add a comment |Â
up vote
1
down vote
up vote
1
down vote
On a much lower level, when a program exits, the OS kernel closes all file descriptors opened by that program. These include network sockets.
On a much lower level, when a program exits, the OS kernel closes all file descriptors opened by that program. These include network sockets.
answered Aug 20 at 11:09
Marius Gedminas
8,11122834
8,11122834
Very interesting. Even more interesting is that, as the accepted answer says, the connection may not be closed if there are no errors... would be nice to explore that...
â machetazo
Aug 20 at 19:50
add a comment |Â
Very interesting. Even more interesting is that, as the accepted answer says, the connection may not be closed if there are no errors... would be nice to explore that...
â machetazo
Aug 20 at 19:50
Very interesting. Even more interesting is that, as the accepted answer says, the connection may not be closed if there are no errors... would be nice to explore that...
â machetazo
Aug 20 at 19:50
Very interesting. Even more interesting is that, as the accepted answer says, the connection may not be closed if there are no errors... would be nice to explore that...
â machetazo
Aug 20 at 19:50
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f51831726%2fpython-what-happens-if-script-stops-while-requests-get-is-executing%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password