How to download a file using HTTP in Java? Example

Hello guys, if you are looking for tutorial to understand about how to download a  file using http in Java to have deep knowledge of java basics then you will definitely learn from this article. uploading and downloading files are common task but many Java developer are not aware about how to complete that. They don't know about HTTP and how to send and receive data using HTTP. Many of them also doesn't know about HttpClient which was added in JDK 11 as well as third party libraries like Apache HttpClient which you can use before JDK 11. Anyway, don't worry, we are again here with new article that is on download file using http will give all basics about the topic to our viewers or readers.  Let's start with an example first.

How to download a file in Java?

Here is an example of how to download a file using Java and the Apache HttpClient library

import java.io.FileOutputStream;

import java.io.IOException;

import java.io.InputStream;

import java.net.URL;

import org.apache.http.HttpEntity;

import org.apache.http.client.methods.CloseableHttpResponse;

import org.apache.http.client.methods.HttpGet;

import org.apache.http.impl.client.CloseableHttpClient;

import org.apache.http.impl.client.HttpClients;

public class FileDownloadExample {

    public static void main(String[] args) {

        CloseableHttpClient httpClient = HttpClients.createDefault();

        HttpGet httpGet = new HttpGet("https://www.example.com/file.zip");

        try (CloseableHttpResponse response = httpClient.execute(httpGet)) {

            HttpEntity entity = response.getEntity();

            if (entity != null) {

                try (InputStream inputStream = entity.getContent();

                    FileOutputStream fileOutputStream = new FileOutputStream("file.zip")) {

                    int read;

                    byte[] buffer = new byte[4096];

                    while ((read = inputStream.read(buffer)) != -1) {

                        fileOutputStream.write(buffer, 0, read);

                    }

                }

            }

        } catch (IOException e) {

            e.printStackTrace();

        } finally {

            try {

                httpClient.close();

            } catch (IOException e) {

                e.printStackTrace();

            }

        }

    }

}

Explanation

The code creates an instance of CloseableHttpClient, creates an instance of HttpGet and executes  it using the httpClient instance. The response from the HTTP request is obtained using the response.getEntity() method and stored in an instance of HttpEntity. 

The content of the HTTP response is then read using the getContent() method of the HttpEntity instance, which returns an InputStream. The content is then written to a file using a FileOutputStream.

It is worth noting that this code downloads the file to the local file system, it does not store it in memory. If the file is very large, this approach will minimize the memory footprint.



Exception Handling:

When downloading a file over HTTP, it is a good idea to handle exceptions and handle errors in a proper manner. In the example code above, exceptions are caught and printed to the console using e.printStackTrace(). This is useful for debugging purposes, but in a production environment, you may want to handle exceptions differently.

For example, you may want to throw a custom exception, log the error message, or display an error message to the user. In the case of a failed download, you may also want to retry the download after a certain period of time or a certain number of times. It's also a good idea to handle different HTTP response codes properly. 

For example, a 404 Not Found error means that the file you're trying to download doesn't exist, while a 500 Internal Server Error means there's a problem on the server side. You can handle these errors by checking the status code of the response, using the response.getStatusLine().getStatusCode() method.

You may also want to set a timeout for the HTTP request to avoid waiting indefinitely if the server is unresponsive or slow. This can be done by setting the timeout values on the RequestConfig of the HttpClient instance.


Example

RequestConfig config = RequestConfig.custom()
.setConnectTimeout(5000)
.setSocketTimeout(5000)
.build();


CloseableHttpClient httpClient = HttpClients.custom()
.setDefaultRequestConfig(config)
.build();

In the code above, the connectTimeout value is set to 5000 milliseconds (5 seconds) and  the socketTimeout value is set to 5000 milliseconds (5 seconds) as well. This means that the request will time out if the server does not respond within 5 seconds.

Additionally, it is also a good idea to handle potential problems with the file system, such as running out of disk space or the file already existing, when writing the downloaded file to disk.

User agent for the HTTP request

You can also set the user agent for the HTTP request. The user agent is a string that identifies the client software and version, and information about the client device and operating system. Setting the user agent can be useful for tracking the usage of your application and identifying any potential compatibility issues.

HttpGet request = new HttpGet("http://example.com/file.zip");
request.addHeader("User-Agent", "MyJavaDownloader/1.0");

In the code above, the user agent string "MyJavaDownloader/1.0" is added as a header to the HttpGet request as a header. You can set any string as the user agent, but it's recommended that you use a string that is descriptive and includes the name and version of your application. 

It's also a good idea to set the Accept-Encoding header to specify which encodings are acceptable in the response. This allows the server to send a compressed response if available, which can result in faster download times.


Example:

request.addHeader("Accept-Encoding", "gzip, deflate");
request.addHeader("Accept-Encoding", "gzip, deflate");

In the code above, the Accept-Encoding header is set to "gzip, deflate," which means that the client is able to handle both gzip and deflate encodings. By setting the user agent and accepting encoding headers, you can improve the performance and compatibility of your download process.

Download Progress

Another important aspect to consider when downloading a file using Java is the progress of the download. You may want to display the download progress to the user, or you may want to implement a mechanism to resume the download if it is interrupted.

To track the progress of the download, you can wrap the input stream in a BufferedInputStream and use the read method to read a certain amount of data from the stream. You can then calculate the percentage of the file that has been downloaded and update the progress indicator.


Example:

long fileSize = response.getEntity().getContentLength();

BufferedInputStream bis = new BufferedInputStream(response.getEntity().getContent());

long totalBytesRead = 0;

int percentCompleted = 0;




while ((bytesRead = bis.read(buffer)) != -1) {

totalBytesRead += bytesRead;

int newPercentCompleted = (int) (totalBytesRead * 100 / fileSize);

if (newPercentCompleted > percentCompleted) {

percentCompleted = newPercentCompleted;

System.out.println("Download progress: " + percentCompleted + "%");

}

fileOutputStream.write(buffer, 0, bytesRead);

}

In the code above, the file size is obtained from the response using the getContentLength method.

The BufferedInputStream is then used to read the data in chunks, and the number of bytes read is accumulated in the totalBytesRead variable.  

The percentage of the file that has been downloaded is calculated as totalBytesRead * 100 / fileSize, and is printed to the console whenever it changes. By tracking the progress of the download, you can provide the user with feedback on the status of the download and implement mechanisms for resuming the download if necessary.

Conclusion
Finally, it's a good idea to properly close the HttpClient and InputStream instances after use. This is done in the example code above by using the try-with-resources statement, which automatically closes the instances even if an exception is thrown. In summary, downloading a file using Java requires careful consideration of error handling, response code handling, resource management, and timeout settings. It is important to handle these issues appropriately to ensure a reliable and efficient download process.