Python requests read timeout

Timeouts in Python requests

In this tutorial, you’ll learn how to use timeouts in the Python requests library, when working with any type of HTTP request being made. By default, the requests library will not time out any request you make, which can result in your programming running indefinitely if a server doesn’t respond.

By the end of this tutorial, you’ll have learned:

• How to set timeouts in requests
• How to set unique timeouts for connecting and reading in Python requests
• How to catch and handle timeout errors in Python requests

How Does Python requests Handle Timeouts?

By default, the Python requests library does not set a timeout for any request it sends. This is true for GET , POST , and PUT requests. While this can prevent unexpected errors, it can result in your request running indefinitely.

Because of this, it’s important to set a timeout to prevent unexpected behavior. Remember, the Python requests library will not timeout by default, unless explicitly instructed.

How to Set a Timeout for Python requests

In order to set a timeout in an HTTP request made via the requests library, you can use the timeout parameter. The parameter accepts either an integer or a floating point value, which describes the time in seconds.

It’s important to note that this behavior is different from many other HTTP request libraries, such as those in JavaScript. In other libraries or languages, this behavior tends to be expressed in milliseconds.

Let’s take a look at an example of how we can send a GET request with a timeout:

# Setting a Timeout on a GET Request with an Integer import requests resp = requests.get('https://datagy.io', timeout=3)

In the example above, we set a timeout of 3 seconds. We used an integer to represent the time of our timeout. If we wanted to be more precise, we could also pass in a floating point value:

# Setting a Timeout on a GET Request with a Floating Point Value import requests resp = requests.get('https://datagy.io', timeout=3.5)

By passing in a single value, we set a timeout for the request. If we wanted to set different timeouts for connecting and reading a request, we can pass in a tuple of values.

How to Set Timeouts for Connecting and Reading in Python requests

In some cases, you’ll want to set different timeouts for making a connection and for reading results. This can easily be done using the timeout parameter in the requests library. Similar to the example above, this can be applied to any type of request being made.

Let’s see how we can pass in different timeout limits for connecting and reading requests in the Python requests library:

# Setting Different Timeouts for Connecting and Reading Requests import requests resp = requests.get('https://datagy.io', timeout=(1, 2))

In the example above, we set the request to timeout after 1 second for connecting and 2 seconds for reading the request.

In the following section, you’ll learn how to catch and handle errors that arise due to requests timing out.

How to Catch and Handle Timeout Errors in Python requests

When applying a timeout, it’s important to note that this is not a time limit on the entire response. Instead, it raises an exception if no bytes have been received on the underlying socket.

If the request does not receive any bytes within the specified timeout limit, a Timeout error is raised. Let’s see what this looks like:

# Raising a Timeout Error in a Python requests GET Request import requests resp = requests.get('https://datagy.io', timeout=0.0001) # Raises: # ConnectTimeoutError(, 'Connection to datagy.io timed out. (connect timeout=0.0001)')

In order to prevent your program from crashing, you need to handle the exception using a try-except block. Let’s see how this can be done:

# Handling a Timeout Error import requests from requests.exceptions import ConnectTimeout try: requests.get('https://datagy.io', timeout=0.0001) except ConnectTimeout: print('Request has timed out') # Returns: # Request has timed out

We can see in the code above that the error was handled safely. In order to do this, we:

1. Imported the error from the exceptions module of the requests library
2. We created a try-except block to handle the ConnectTimeout error.

Frequently Asked Questions

None. There is no default timeout for Python requests, unless explicitly set using the timeout parameter.

You set a timeout (in seconds) using the timeout= parameter when making HTTP requests in the Python requests library.

While there is no best set value for timeouts for HTTP requests made in Python, a good practice is to set them under 500ms. This allows your application to provide a better user experience and to process more requests.

Conclusion

In this tutorial, you learned how to handle timeouts in the Python requests library. You first learned how the Python requests library handles timeouts. Then, you learned how to set timeouts when making HTTP requests, both using integers and floating point values. Then, you learned how to specify specific timeouts for connecting and reading requests. Finally, you learned how to handle timeout exceptions in the Python requests library.

Additional Resources

To learn more about related topics, check out the tutorials below:

Источник

Timeout in Python requests – Everything you need to know

Requests is the de-facto standard when it comes to making HTTP requests in Python, especially since Python 3. The open-source library abstracts away the complexity of managing network connections and make sending HTTP requests a breeze.

Most of you may already know how to send HTTP POST, GET, as well as other type of HTTP requests. In this article, we will guide you through how to set timeout in requests , as well as managing exceptions.

Timeouts in Python requests

You can tell requests library to stop waiting for a response after a given amount of time by passing a number to the timeout parameter. If the requests library does not receive response in x seconds, it will raise a Timeout error.

It’s a best practice that production code should use this parameter in all network requests. Failure to do so can cause your program to hang indefinitely. If no timeout is specified explicitly, requests do not time out.

requests.get('https://example.com/', timeout=10) # OUTPUT Traceback (most recent call last): File "", line 1, in requests.exceptions.Timeout: HTTPConnectionPool(host='example.com', port=80): Request timed out. (timeout=10)Code language: PHP (php)

Catch timeout exception

Once a timeout value has been set, every request that doesn’t receive a response in the specified timeframe will raise a Timeout error. It’s important to handle this exception, otherwise your program would be terminated.

In order to catch Timeout errors in requests , you have to import the exception itself using from requests.exceptions import Timeout .

import requests from requests.exceptions import Timeout try: requests.get('https://www.example.com, timeout=10) except Timeout: print('Timeout has been raised.')Code language: PHP (php)

Advanced timeout handling

Most requests to external servers should have a timeout attached, in case the server is not responding in a timely manner. By default, requests do not time out unless a timeout value is set explicitly. Without a timeout, your code may hang for minutes or more.

The connect timeout is the number of seconds Requests will wait for your client to establish a connection to a remote machine (corresponding to the connect()) call on the socket. It’s a good practice to set connect timeouts to slightly larger than a multiple of 3, which is the default TCP packet retransmission window.

Once your client has connected to the server and sent the HTTP request, the read timeout is the number of seconds the client will wait for the server to send a response. (Specifically, it’s the number of seconds that the client will wait between bytes sent from the server. In 99.9% of cases, this is the time before the server sends the first byte).

If you specify a single value for the timeout, like this:

r = requests.get('https://github.com', timeout=5)Code language: JavaScript (javascript)

The timeout value will be applied to both the connect and the read timeouts. Specify a tuple if you would like to set the values separately:

r = requests.get('https://github.com', timeout=(3.05, 27))Code language: JavaScript (javascript)

Request timeout in async coroutines

Since there are a lot of beginners asked how to use requests in asynchronous programs or coroutines, we’ve made this section dedicated to answering just that.

requests is a blocking library, which means it should not be used in asynchronous coroutines. The proper libraries to use in this context is aiohttp and async-timeout. aiohttp is an asynchronous HTTP client/server framework, while async-timeout is a asyncio-compatible timeout context manager that can be used to wrap any coroutines inside a “timeout” context manager.

Источник