IBM Cloud Object Storage

IBM Cloud Object Storage

• 1.
  Nagesh Ramamoorthy
• 2.
  Agenda • ICOS Overview •Storage Classes • Resiliency Options • End points • Access Policies • Service credentials and HMAC credentials • Firewalls & Encryption • Aspera High Speed Transfer • Lifecycle Rules: Expiration and Archival • Immutable Object Storage • IBM Cloud SQL Query
• 3.
  ICOS Overview • Formerlyknown as Cleversafe. • IBM COS supports objects up to 10 TB, and maximum of 100 buckets. • S3 API support is available in order to provide compatibility to standalone clients for AWS S3 storage. • IBM COS is IAM enabled. • We can enable Activity tracker based API logging for Each bucket level management and data events
• 4.
  Storage Classes Four storageClasses: • Standard :Used for active workloads , no retrieval fee • Vault: Used for Cold data and retrieval fee applicable • Cold Vault: Used for cold data , not accessed for more than 90 days . More retrieval fee applicable • Flex: Used for dynamic workloads with no predictable usage patterns
• 5.
  Resiliency Options Three typesof resiliency/replication provided: Cross-Region ( Data replicated across three regions in a geography) Regional ( Data is replicated across three AZs in a region) Single Datacenter ( Data is replicated across multiple servers in the same location)
• 6.
  End Points • ICOSsupports private and public end points. • VPC endpoints can connect to ICOS using a separate direct end points privately . • There are different end points for Regional , Cross-regional and datacenter locations. • Regional End Points for US-South Region example: Public: s3.us-south.cloud-object-storage.appdomain.cloud Private: s3.private.us-south.cloud-object-storage.appdomain.cloud Direct: s3.direct.us-south.cloud-object-storage.appdomain.cloud
• 7.
  Access Policy • Everyuser that accesses the IBM® Cloud Object Storage service in your account must be assigned an access policy with an IAM user role pre-defined ( Platform management and service access) • There is no bucket resource level permission option other than through IAM method. • Using IAM access policies , permissions can be granted at individual bucket level. • Public access can be granted by clicking on "access policy" inside bucket configuration
• 8.
  Service and HMACcredentials • A service credential provides the necessary information to connect an application to Object Storage packaged in a JSON document. • "Service credentials" option under object storage tab allows to create service id and associate privileges for all the buckets in the storage service along with end point details in a json document. • When a service credential is created, the underlying Service ID is granted a role on the entire instance of Object Storage. • If the intention that the credential be used to grant, access to a subset of buckets and not the entire instance, this policy needs to be edited. • HMAC credentials contains an access key and secret access key which is compatible to AWS S3 API. • HMAC credentials can be generated as part of "service credentials" option
• 9.
  Firewalls and Encryption •We can set up firewall by allowing certain limited number of IPs to access the bucket. • Once the firewall is setup , other IBM coud services can't access the bucket privately. • The objects are encrypted by default at rest with automatic provider side Advanced Encryption Standard (AES) 256-bit encryption and Secure Hash Algorithm (SHA)- 256 hash. • IBM Cloud Object storage provides option to encrypt through customer provided keys which is called server side encryption with customer provided keys (SSE-C) and also through SSE-KP (Server side encryption with IBM Key protect)
• 10.
  Aspera High-Speed Transfer •Aspera High Speed transfer allows transfers larger than 200 MB through console using proprietary FASP ( Fast and secure Protocol) • Aspera High Speed transfer requires either a browser plug-in or a desktop agent • Aspera High Speed transfer supports Java and Python SDKs • Aspera High Speed transfer supports windows, Ubuntu Linux and Mac OS agents
• 11.
  Lifecycle Rules: Expiration, Archival • Expiration rule makes the objects deleted automatically after given number of days from object creation. • IBM Cloud object storage archive is a low cost option for data that is rarely accessed. • You can transition data from any storage class ( Standard , Vault, Cold Vault ,Flex) to Archive. • For immediate archival , the archival time should be set to 0 days. • To access the data that is archived , it should be restored by specifying the period of which the object should be kept in the original class. • The restoration duration can be up to 12 hours • Together Expiration and Archive policies , we can set up to 1000 life cycle policies
• 12.
  Immutable Object Storage •Immutable Object Storage preserves electronic records and maintains data integrity. • Retention policies ensure that data is stored in a WORM (Write-Once-Read-Many), non- erasable and non-rewritable manner. • Retention Policies allows prevention of deletion of object within specified time. • Retention policies once enabled, can't be disabled • Retention policy can be set while uploading an object as well but the specified value should be within minimum and maximum value set at the bucket level. • The default retention period can be set at the bucket configuration. • Enabling "Permanent retention" at bucket level ,never allows objects deletion
• 13.
  IBM Cloud SQL •IBM Cloud SQL is a fully managed service which allows to run "SELECT" statements on object storage files of ORC, CSV, JSON format. • The query results are stored in a CSV file in the object storage. • Actions with Cloud SQL such as CREATE, DELETE, INSERT, and UPDATE are not possible.