Amazon Simple Storage Service (Amazon S3) is object storage built to store and retrieve any amount of data from web or mobile. Amazon S3 is designed to scale computing easier for developers. For starting, in the tutorial, JavaSampleApproach show you how to create a SpringBoot Amazon S3 application.
Related post:
– Amazon S3 – How to upload/download large files to S3 with SpringBoot Amazon S3 MultipartFile application
I. Technologies
– Java 8
– Maven 3.6.1
– Spring Tool Suite: Version 3.8.4.RELEASE
– Spring Boot: 1.5.4.RELEASE
– Amazon S3
II. SpringBoot Amazon S3
In the tutorial, JavaSampleApproach will setup an Amazon S3 bucket, then use SpringBoot application with aws-java-sdk to upload/download files to/from S3.
– For init an AmazonS3 client, we use:
BasicAWSCredentials awsCreds = new BasicAWSCredentials(awsId, awsKey); AmazonS3 s3Client = AmazonS3ClientBuilder.standard() .withRegion(Regions.fromName(region)) .withCredentials(new AWSStaticCredentialsProvider(awsCreds)) .build();
– For upload S3 objects, we use:
public PutObjectResult putObject(PutObjectRequest putObjectRequest) throws SdkClientException, AmazonServiceException;
– For download S3 objects, we use:
public S3Object getObject(GetObjectRequest getObjectRequest) throws SdkClientException, AmazonServiceException;
III. Practice
In the tutorial, we will setup an Amazon S3 bucket, an IAM user and create a SpringBoot application to upload/download files to/from S3.
Step to do:
Setup Amazon S3
– Sign Up
– Create a Bucket
– Create an IAM user
SpringBoot Amazon S3 application
– Create SpringBoot project
– Set Amazon S3 client
– Implement S3 download/upload services
– Implement simple client
– Run and check results
1. Setup Amazon S3
1.1 Sign Up
Go to https://aws.amazon.com/s3/
Choose Get started with Amazon S3.
Follow the instructions on the screen for sign up.
-> AWS will notify you by email when your account is active and available for you to use.
1.2 Create a Bucket
Now, your account is ready to use with AWS.
-> Sign in to AWS: https://console.aws.amazon.com/s3
Choose Create bucket, input information for creating bucket:
Press Create -> result:
1.3 Create an IAM user
Go to https://console.aws.amazon.com/iam/
In the navigation pane, choose Users and then choose Add user.
Input User name, choose Programmatic access for Access type:
Press Next: Permissions button -> go to Set permissions for jsa-user screen.
Now, choose Attach existing policies directly -> filter policy type s3, then check AmazonS3FullAccess:
Press Next: Review:
Press Create user:
Press Download .csv for {Access key ID, Secret access key}.
2. SpringBoot Amazon S3 application
2.1 Create SpringBoot project
Using SpringToolSuite to create a SpringBoot project. Then add needed dependency aws-java-sdk
:
com.amazonaws aws-java-sdk 1.11.106
2.2 Set Amazon S3 client
Create a S3Config.java file for setup AmazonS3 client:
package com.javasampleapproach.s3.config; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import com.amazonaws.auth.AWSStaticCredentialsProvider; import com.amazonaws.auth.BasicAWSCredentials; import com.amazonaws.regions.Regions; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3ClientBuilder; @Configuration public class S3Config { @Value("${jsa.aws.access_key_id}") private String awsId; @Value("${jsa.aws.secret_access_key}") private String awsKey; @Value("${jsa.s3.region}") private String region; @Bean public AmazonS3 s3client() { BasicAWSCredentials awsCreds = new BasicAWSCredentials(awsId, awsKey); AmazonS3 s3Client = AmazonS3ClientBuilder.standard() .withRegion(Regions.fromName(region)) .withCredentials(new AWSStaticCredentialsProvider(awsCreds)) .build(); return s3Client; } }
Open application.properties, add aws configuration:
jsa.aws.access_key_id=AKIAIP6OYZ7NICKOQSLQ jsa.aws.secret_access_key=***(change the key) jsa.s3.bucket=jsa-s3-bucket jsa.s3.region=us-east-1 jsa.s3.uploadfile=C:\\s3\\jsa-s3-upload-file.txt jsa.s3.key=jsa-s3-upload-file.txt
Note: Keep private for secret_access_key.
2.3 Implement S3 download/upload services
Create S3Services interface:
package com.javasampleapproach.s3.services; public interface S3Services { public void downloadFile(String keyName); public void uploadFile(String keyName, String uploadFilePath); }
Implement S3Services interface with S3ServicesImpl:
package com.javasampleapproach.s3.services.impl; import java.io.File; import java.io.IOException; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Value; import org.springframework.stereotype.Service; import com.amazonaws.AmazonClientException; import com.amazonaws.AmazonServiceException; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.model.GetObjectRequest; import com.amazonaws.services.s3.model.PutObjectRequest; import com.amazonaws.services.s3.model.S3Object; import com.javasampleapproach.s3.services.S3Services; import com.javasampleapproach.s3.util.Utility; @Service public class S3ServicesImpl implements S3Services { private Logger logger = LoggerFactory.getLogger(S3ServicesImpl.class); @Autowired private AmazonS3 s3client; @Value("${jsa.s3.bucket}") private String bucketName; @Override public void downloadFile(String keyName) { try { System.out.println("Downloading an object"); S3Object s3object = s3client.getObject(new GetObjectRequest(bucketName, keyName)); System.out.println("Content-Type: " + s3object.getObjectMetadata().getContentType()); Utility.displayText(s3object.getObjectContent()); logger.info("===================== Import File - Done! ====================="); } catch (AmazonServiceException ase) { logger.info("Caught an AmazonServiceException from GET requests, rejected reasons:"); logger.info("Error Message: " + ase.getMessage()); logger.info("HTTP Status Code: " + ase.getStatusCode()); logger.info("AWS Error Code: " + ase.getErrorCode()); logger.info("Error Type: " + ase.getErrorType()); logger.info("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { logger.info("Caught an AmazonClientException: "); logger.info("Error Message: " + ace.getMessage()); } catch (IOException ioe) { logger.info("IOE Error Message: " + ioe.getMessage()); } } @Override public void uploadFile(String keyName, String uploadFilePath) { try { File file = new File(uploadFilePath); s3client.putObject(new PutObjectRequest(bucketName, keyName, file)); logger.info("===================== Upload File - Done! ====================="); } catch (AmazonServiceException ase) { logger.info("Caught an AmazonServiceException from PUT requests, rejected reasons:"); logger.info("Error Message: " + ase.getMessage()); logger.info("HTTP Status Code: " + ase.getStatusCode()); logger.info("AWS Error Code: " + ase.getErrorCode()); logger.info("Error Type: " + ase.getErrorType()); logger.info("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { logger.info("Caught an AmazonClientException: "); logger.info("Error Message: " + ace.getMessage()); } } }
For support displayText, we create an Utility
class with a function Utility.displayText(InputStream input)
:
package com.javasampleapproach.s3.util; import java.io.BufferedReader; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; public class Utility { public static void displayText(InputStream input) throws IOException{ // Read one text line at a time and display. BufferedReader reader = new BufferedReader(new InputStreamReader(input)); while (true) { String line = reader.readLine(); if (line == null) break; System.out.println(" " + line); } } }
2.4 Implement simple client
Using S3Services
, We implement a client for upload/download files:
package com.javasampleapproach.s3; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Value; import org.springframework.boot.CommandLineRunner; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import com.javasampleapproach.s3.services.S3Services; @SpringBootApplication public class SpringS3AmazonApplication implements CommandLineRunner{ @Autowired S3Services s3Services; @Value("${jsa.s3.uploadfile}") private String uploadFilePath; @Value("${jsa.s3.key}") private String downloadKey; public static void main(String[] args) { SpringApplication.run(SpringS3AmazonApplication.class, args); } @Override public void run(String... args) throws Exception { System.out.println("---------------- START UPLOAD FILE ----------------"); s3Services.uploadFile("jsa-s3-upload-file.txt", uploadFilePath); System.out.println("---------------- START DOWNLOAD FILE ----------------"); s3Services.downloadFile(downloadKey); } }
2.5 Run and check results
– Create a file C:\\s3\\jsa-s3-upload-file.txt
with content:
JSA uploads a file to S3 Amazon!
– Build and Run the SpringBoot project with commandlines {mvn clean install
, mvn spring-boot:run
}.
-> Upload file:
-> Download file, Logs:
---------------- START DOWNLOAD FILE ---------------- Downloading an object Content-Type: text/plain JSA uploads a file to S3 Amazon!