How to Fill an OSS Bucket Rapidly

As the end of the year approaches, it's once again time for faking.

Mission

Fill an OSS bucket with 600T data.

Solution

Create bucket

Login CUCloud and create a bucket. Here it's named test.

Get credentials

Get aws_access_key_id and aws_secret_access_key from Cloud.

Install AWS

In my Manjaro Linux, I install AWS by the command below:

sudo pamac install aws-cli-v2

Set AWS

Run commmand below to set AWS:

aws configure

• Input aws_access_key_id and aws_secret_access_key as above.
• Set region according the OSS region, usually it's cn-xxxx-1.
• Set output to json.

Upload seed

Use AWS ❌

Create a 10G file and upload it as a seed.

dd if=/dev/zero bs=1M count=10240 status=progress | aws s3 cp - s3://test/seed_10G.img --endpoint-url https://obs-xxxx.cucloud.cn --no-verify-ssl

NOTE: Due to the Wildcard Certificate, I have to add --no-verify-ssl to avoid the error message: hostname obs-xxxx.cucloud.cn' doesn't match '*.obs-xxxx.cucloud.cn'

Use Python ✅

But, when I upload as above, it fails with error MissingContentLength. That means this OSS requires Content-Length. So I have to write a Python script:

import boto3
import os

# --- 配置信息 ---
endpoint = 'https://obs-xxxx.cucloud.cn'
bucket_name = 'test'
file_name = 'seed_10G.img'

# 如果你之前运行过 aws configure，boto3 会自动读取 AK/SK。
# 如果没有，请手动填入：
access_key = 'ACCESS_KEY'
secret_key = 'SECRET_KEY'
# ----------------

# 1. 创建 S3 客户端
s3 = boto3.client('s3',
    endpoint_url=endpoint,
    aws_access_key_id=access_key,
    aws_secret_access_key=secret_key,
    verify=False # 禁用 SSL 验证，以兼容你之前使用的 --no-verify-ssl 方式
)

# 2. 获取文件大小 (boto3 内部会使用它)
try:
    file_size = os.path.getsize(file_name)
except FileNotFoundError:
    print(f"Error: File '{file_name}' not found.")
    exit()

# 3. 上传文件
try:
    print(f"Starting upload for {file_name} ({file_size} bytes) to {endpoint}...")
    # put_object 方法会读取文件对象，并自动添加 Content-Length HTTP Header
    with open(file_name, 'rb') as data:
        s3.put_object(
            Bucket=bucket_name,
            Key=file_name,
            Body=data,
            ContentLength=file_size # 显式传递，增强兼容性
        )
    print("✅ Upload successful!")

except Exception as e:
    print(f"❌ Upload failed with error: {e}")

NOTE: Create the seed file by using fallocate -l 10G seed_10G.img.

Duplicate seed

After the seed is uploaded, run the command below to duplicate it:

seq 1 60000 | xargs -P 20 -I {} aws s3 cp s3://test/seed_10G.img s3://test/copy_{}.img --endpoint-url https://obs-xxxx.cucloud.cn --no-verify-ssl 2>/dev/null

The operation is running inside of the bucket, so it is very rapid.

And, here's an alternative way by using uuidgen to create random filename:

while true; do uuidgen | xargs -P 20 -I {} aws s3 cp s3://test/seed_10G s3://test/{}.img --endpoint-url https://obs-zjsx.cucloud.cn --no-verify-ssl; done