Original skill document
Rendered from the source repository. Headings, examples, code, tables, links, and referenced images are preserved.
AWS SDK for Java 2.x - Amazon S3
Overview
Provides patterns for S3 operations: bucket management, object upload/download with multipart support, presigned URLs, S3 Transfer Manager, and S3-specific configurations using AWS SDK for Java 2.x.
When to Use
- Creating, listing, or deleting S3 buckets with proper configuration
- Uploading or downloading objects from S3 with metadata and encryption
- Working with multipart uploads for large files (>100MB) with error handling
- Generating presigned URLs for temporary access to S3 objects
- Copying or moving objects between S3 buckets with metadata preservation
- Setting object metadata, storage classes, and access controls
- Implementing S3 Transfer Manager for optimized file transfers
- Integrating S3 with Spring Boot applications for cloud storage
Quick Reference
| Operation | Method | Notes |
|---|---|---|
| Create bucket | createBucket() | Wait with waiter().waitUntilBucketExists() |
| Upload object | putObject() | Use RequestBody.fromFile() |
| Download object | getObject() | Streams to file or memory |
| Delete objects | deleteObjects() | Batch up to 1000 keys |
| Presigned URL | presigner.presignGetObject() | Max 7 days expiration |
Storage Classes
| Class | Use Case |
|---|---|
STANDARD | Frequently accessed data |
STANDARD_IA | Infrequently accessed data |
GLACIER | Long-term archive |
INTELLIGENT_TIERING | Automatic cost optimization |
Instructions
1. Add Dependencies
xml
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
<version>2.20.0</version>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3-transfer-manager</artifactId>
<version>2.20.0</version>
</dependency>2. Create S3 Client
java
S3Client s3Client = S3Client.builder()
.region(Region.US_EAST_1)
.build();
// With retry logic
S3Client s3Client = S3Client.builder()
.region(Region.US_EAST_1)
.overrideConfiguration(b -> b
.retryPolicy(RetryPolicy.builder()
.numRetries(3)
.build()))
.build();3. Create Bucket
java
CreateBucketRequest request = CreateBucketRequest.builder()
.bucket(bucketName)
.build();
s3Client.createBucket(request);
// Wait until ready
s3Client.waiter().waitUntilBucketExists(
HeadBucketRequest.builder().bucket(bucketName).build()
);4. Upload Object
java
PutObjectRequest request = PutObjectRequest.builder()
.bucket(bucketName)
.key(key)
.contentType("application/pdf")
.serverSideEncryption(ServerSideEncryption.AES256)
.storageClass(StorageClass.STANDARD_IA)
.build();
s3Client.putObject(request, RequestBody.fromFile(Paths.get(filePath)));
// Validate upload completion
HeadObjectResponse headResp = s3Client.headObject(HeadObjectRequest.builder()
.bucket(bucketName)
.key(key)
.build());5. Download Object
java
GetObjectRequest request = GetObjectRequest.builder()
.bucket(bucketName)
.key(key)
.build();
s3Client.getObject(request, Paths.get(destPath));6. Generate Presigned URL
java
try (S3Presigner presigner = S3Presigner.create()) {
GetObjectRequest getRequest = GetObjectRequest.builder()
.bucket(bucketName)
.key(key)
.build();
GetObjectPresignRequest presignRequest = GetObjectPresignRequest.builder()
.signatureDuration(Duration.ofMinutes(10))
.getObjectRequest(getRequest)
.build();
String url = presigner.presignGetObject(presignRequest).url().toString();
}7. Use Transfer Manager (Large Files)
java
try (S3TransferManager tm = S3TransferManager.create()) {
UploadFileRequest request = UploadFileRequest.builder()
.putObjectRequest(req -> req.bucket(bucketName).key(key))
.source(Paths.get(filePath))
.build();
FileUpload upload = tm.uploadFile(request);
CompletedFileUpload result = upload.completionFuture().join();
}Best Practices
Performance
- Use S3 Transfer Manager: Automatic multipart uploads for files >100MB
- Reuse S3 Client: Clients are thread-safe; reuse throughout application
- Enable async operations: Use
S3AsyncClientfor I/O-bound operations - Configure timeouts: Set appropriate timeouts for large file operations
Security
- Use temporary credentials: IAM roles or AWS STS for short-lived tokens
- Enable encryption: Use AES-256 or AWS KMS for sensitive data
- Use presigned URLs: Avoid exposing credentials with temporary access
- Validate metadata: Sanitize user-provided metadata
Error Handling
- Implement retry logic: Exponential backoff for network operations
- Handle throttling: Proper handling of 429 responses
- Clean up failures: Abort failed multipart uploads
Cost Optimization
- Use appropriate storage classes: STANDARD, STANDARDIA, INTELLIGENTTIERING
- Implement lifecycle policies: Automatic transition/expiration
- Minimize API calls: Use batch operations when possible
Constraints and Warnings
- Object Size: Single PUT limited to 5GB; use multipart for larger files
- Bucket Names: Must be globally unique across all AWS accounts
- Object Immutability: Objects cannot be modified; must be replaced entirely
- Eventual Consistency: List operations may have slight delays after uploads
- Presigned URLs: Maximum expiration time is 7 days
- Multipart Uploads: Parts must be at least 5MB except last part
Examples
Complete Upload Workflow with Validation
java
// 1. Upload with validation
PutObjectRequest putRequest = PutObjectRequest.builder()
.bucket(bucketName)
.key(key)
.contentType(contentType)
.build();
s3Client.putObject(putRequest, RequestBody.fromFile(Paths.get(filePath)));
// 2. Validate with headObject
HeadObjectResponse headResp = s3Client.headObject(HeadObjectRequest.builder()
.bucket(bucketName)
.key(key)
.build());
// 3. Verify metadata
long fileSize = Files.size(Paths.get(filePath));
if (headResp.contentLength() != fileSize) {
throw new IllegalStateException("Upload size mismatch");
}Multipart Upload with Abort-on-Failure
java
// 1. Initiate multipart upload
CreateMultipartUploadRequest createRequest = CreateMultipartUploadRequest.builder()
.bucket(bucketName)
.key(key)
.build();
CreateMultipartUploadResponse multipartUpload = s3Client.createMultipartUpload(createRequest);
String uploadId = multipartUpload.uploadId();
try {
// 2. Upload parts
List<CompletedPart> parts = new ArrayList<>();
int partNumber = 1;
byte[] fileBytes = Files.readAllBytes(Paths.get(filePath));
int chunkSize = 5 * 1024 * 1024; // 5MB minimum
for (int offset = 0; offset < fileBytes.length; offset += chunkSize) {
int length = Math.min(chunkSize, fileBytes.length - offset);
UploadPartRequest uploadPartRequest = UploadPartRequest.builder()
.bucket(bucketName)
.key(key)
.uploadId(uploadId)
.partNumber(partNumber)
.build();
UploadPartResponse partResponse = s3Client.uploadPart(uploadPartRequest,
RequestBody.fromBytes(Arrays.copyOfRange(fileBytes, offset, offset + length)));
parts.add(CompletedPart.builder()
.partNumber(partNumber)
.eTag(partResponse.eTag())
.build());
partNumber++;
}
// 3. Complete multipart upload
CompleteMultipartUploadRequest completeRequest = CompleteMultipartUploadRequest.builder()
.bucket(bucketName)
.key(key)
.uploadId(uploadId)
.multipartUpload(CompletedMultipartUpload.builder().parts(parts).build())
.build();
s3Client.completeMultipartUpload(completeRequest);
} catch (Exception e) {
// 4. Abort on failure
AbortMultipartUploadRequest abortRequest = AbortMultipartUploadRequest.builder()
.bucket(bucketName)
.key(key)
.uploadId(uploadId)
.build();
s3Client.abortMultipartUpload(abortRequest);
throw new RuntimeException("Upload failed, cleanup performed", e);
}References
- [references/s3-client-setup.md](references/s3-client-setup.md) — Client configuration and basic operations
- [references/s3-object-operations.md](references/s3-object-operations.md) — Advanced object operations
- [references/s3-transfer-patterns.md](references/s3-transfer-patterns.md) — Transfer Manager and multipart uploads
- [references/s3-spring-boot-integration.md](references/s3-spring-boot-integration.md) — Spring Boot integration patterns
- AWS S3 Developer Guide
- AWS SDK for Java 2.x S3 API
Related Skills
aws-sdk-java-v2-core- Core AWS SDK patterns and configurationspring-boot-dependency-injection- Spring dependency injection patterns
