Amazon S3 – Upload/Download files with SpringBoot Amazon S3 application.

Amazon Simple Storage Service (Amazon S3) is object storage built to store and retrieve any amount of data from web or mobile. Amazon S3 is designed to scale computing easier for developers. For starting, in the tutorial, JavaSampleApproach show you how to create a SpringBoot Amazon S3 application.

Related post:
– Amazon S3 – How to upload/download large files to S3 with SpringBoot Amazon S3 MultipartFile application

I. Technologies

– Java 8
– Maven 3.6.1
– Spring Tool Suite: Version 3.8.4.RELEASE
– Spring Boot: 1.5.4.RELEASE
– Amazon S3

II. SpringBoot Amazon S3

In the tutorial, JavaSampleApproach will setup an Amazon S3 bucket, then use SpringBoot application with aws-java-sdk to upload/download files to/from S3.

– For init an AmazonS3 client, we use:

BasicAWSCredentials awsCreds = new BasicAWSCredentials(awsId, awsKey);
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
			.withRegion(Regions.fromName(region))
			.withCredentials(new AWSStaticCredentialsProvider(awsCreds))
			.build();

– For upload S3 objects, we use:

public PutObjectResult putObject(PutObjectRequest putObjectRequest)
            throws SdkClientException, AmazonServiceException;

– For download S3 objects, we use:

public S3Object getObject(GetObjectRequest getObjectRequest)
            throws SdkClientException, AmazonServiceException;

III. Practice

In the tutorial, we will setup an Amazon S3 bucket, an IAM user and create a SpringBoot application to upload/download files to/from S3.

Step to do:
Setup Amazon S3
– Sign Up
– Create a Bucket
– Create an IAM user

SpringBoot Amazon S3 application
– Create SpringBoot project
– Set Amazon S3 client
– Implement S3 download/upload services
– Implement simple client
– Run and check results

1. Setup Amazon S3

1.1 Sign Up

Go to https://aws.amazon.com/s3/

Choose Get started with Amazon S3.

Follow the instructions on the screen for sign up.
-> AWS will notify you by email when your account is active and available for you to use.

1.2 Create a Bucket

Now, your account is ready to use with AWS.
-> Sign in to AWS: https://console.aws.amazon.com/s3

Choose Create bucket, input information for creating bucket:

Press Create -> result:

1.3 Create an IAM user

Go to https://console.aws.amazon.com/iam/
In the navigation pane, choose Users and then choose Add user.

Input User name, choose Programmatic access for Access type:

Press Next: Permissions button -> go to Set permissions for jsa-user screen.
Now, choose Attach existing policies directly -> filter policy type s3, then check AmazonS3FullAccess:

Press Next: Review:

Press Create user:

Press Download .csv for {Access key ID, Secret access key}.

2. SpringBoot Amazon S3 application

2.1 Create SpringBoot project

Using SpringToolSuite to create a SpringBoot project. Then add needed dependency aws-java-sdk:

  com.amazonaws
  aws-java-sdk
  1.11.106

2.2 Set Amazon S3 client

Create a S3Config.java file for setup AmazonS3 client:

package com.javasampleapproach.s3.config;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;

@Configuration
public class S3Config {
	@Value("${jsa.aws.access_key_id}")
	private String awsId;

	@Value("${jsa.aws.secret_access_key}")
	private String awsKey;
	
	@Value("${jsa.s3.region}")
	private String region;

	@Bean
	public AmazonS3 s3client() {
		
		BasicAWSCredentials awsCreds = new BasicAWSCredentials(awsId, awsKey);
		AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
								.withRegion(Regions.fromName(region))
		                        .withCredentials(new AWSStaticCredentialsProvider(awsCreds))
		                        .build();
		
		return s3Client;
	}
}

Open application.properties, add aws configuration:

jsa.aws.access_key_id=AKIAIP6OYZ7NICKOQSLQ
jsa.aws.secret_access_key=***(change the key)
jsa.s3.bucket=jsa-s3-bucket
jsa.s3.region=us-east-1

jsa.s3.uploadfile=C:\\s3\\jsa-s3-upload-file.txt
jsa.s3.key=jsa-s3-upload-file.txt

Note: Keep private for secret_access_key.

2.3 Implement S3 download/upload services

Create S3Services interface:

package com.javasampleapproach.s3.services;

public interface S3Services {
	public void downloadFile(String keyName);
	public void uploadFile(String keyName, String uploadFilePath);
}

Implement S3Services interface with S3ServicesImpl:

package com.javasampleapproach.s3.services.impl;

import java.io.File;
import java.io.IOException;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;

import com.amazonaws.AmazonClientException;
import com.amazonaws.AmazonServiceException;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.PutObjectRequest;
import com.amazonaws.services.s3.model.S3Object;
import com.javasampleapproach.s3.services.S3Services;
import com.javasampleapproach.s3.util.Utility;

@Service
public class S3ServicesImpl implements S3Services {
	
	private Logger logger = LoggerFactory.getLogger(S3ServicesImpl.class);
	
	@Autowired
	private AmazonS3 s3client;

	@Value("${jsa.s3.bucket}")
	private String bucketName;

	@Override
	public void downloadFile(String keyName) {
		
		try {
			
            System.out.println("Downloading an object");
            S3Object s3object = s3client.getObject(new GetObjectRequest(bucketName, keyName));
            System.out.println("Content-Type: "  + s3object.getObjectMetadata().getContentType());
            Utility.displayText(s3object.getObjectContent());
            logger.info("===================== Import File - Done! =====================");
            
        } catch (AmazonServiceException ase) {
        	logger.info("Caught an AmazonServiceException from GET requests, rejected reasons:");
			logger.info("Error Message:    " + ase.getMessage());
			logger.info("HTTP Status Code: " + ase.getStatusCode());
			logger.info("AWS Error Code:   " + ase.getErrorCode());
			logger.info("Error Type:       " + ase.getErrorType());
			logger.info("Request ID:       " + ase.getRequestId());
        } catch (AmazonClientException ace) {
        	logger.info("Caught an AmazonClientException: ");
            logger.info("Error Message: " + ace.getMessage());
        } catch (IOException ioe) {
        	logger.info("IOE Error Message: " + ioe.getMessage());
		}
	}

	@Override
	public void uploadFile(String keyName, String uploadFilePath) {
		
		try {
			
			File file = new File(uploadFilePath);
	        s3client.putObject(new PutObjectRequest(bucketName, keyName, file));
	        logger.info("===================== Upload File - Done! =====================");
	        
		} catch (AmazonServiceException ase) {
			logger.info("Caught an AmazonServiceException from PUT requests, rejected reasons:");
			logger.info("Error Message:    " + ase.getMessage());
			logger.info("HTTP Status Code: " + ase.getStatusCode());
			logger.info("AWS Error Code:   " + ase.getErrorCode());
			logger.info("Error Type:       " + ase.getErrorType());
			logger.info("Request ID:       " + ase.getRequestId());
        } catch (AmazonClientException ace) {
            logger.info("Caught an AmazonClientException: ");
            logger.info("Error Message: " + ace.getMessage());
        }
	}

}

For support displayText, we create an Utility class with a function Utility.displayText(InputStream input):

package com.javasampleapproach.s3.util;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;

public class Utility {
	
	public static void displayText(InputStream input) throws IOException{
		// Read one text line at a time and display.
        BufferedReader reader = new BufferedReader(new InputStreamReader(input));
        while (true) {
            String line = reader.readLine();
            if (line == null) break;
            System.out.println("    " + line);
        }
	}
}

2.4 Implement simple client

Using S3Services, We implement a client for upload/download files:

package com.javasampleapproach.s3;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

import com.javasampleapproach.s3.services.S3Services;

@SpringBootApplication
public class SpringS3AmazonApplication implements CommandLineRunner{

	
	@Autowired
	S3Services s3Services;
	
	@Value("${jsa.s3.uploadfile}")
	private String uploadFilePath;
	
	@Value("${jsa.s3.key}")
	private String downloadKey;
	
	public static void main(String[] args) {
		SpringApplication.run(SpringS3AmazonApplication.class, args);
	}

	@Override
	public void run(String... args) throws Exception {
		System.out.println("---------------- START UPLOAD FILE ----------------");
		s3Services.uploadFile("jsa-s3-upload-file.txt", uploadFilePath);
		System.out.println("---------------- START DOWNLOAD FILE ----------------");
		s3Services.downloadFile(downloadKey);
	}
}

2.5 Run and check results

– Create a file C:\\s3\\jsa-s3-upload-file.txt with content:

JSA uploads a file to S3 Amazon!

– Build and Run the SpringBoot project with commandlines {mvn clean install, mvn spring-boot:run}.

-> Upload file:

-> Download file, Logs:

---------------- START DOWNLOAD FILE ----------------
Downloading an object
Content-Type: text/plain
    JSA uploads a file to S3 Amazon!

IV. Sourcecode

SpringS3Amazon