Source:Backup to S3 Cloud Storage: Difference between revisions

From SEPsesam
m (spaces)
(Added S3-compatible and local folder size info (KRI/RS))
Line 43: Line 43:


==== Multiple media pools ====
==== Multiple media pools ====
With Si3 NG, you can create multiple [[Special:MyLanguage/Configuring_a_Media_Pool|media pools]] to provide scalability and granularity of backup jobs.  You can create separate media pools, e.g., for daily differential and incremental and weekly full backup jobs, or for migration and replication, and you can set a different retention time for each media pool.
With Si3 NG, you can create multiple [[Special:MyLanguage/Configuring_a_Media_Pool|media pools]] to provide scalability and granularity of backup jobs.  You can create separate media pools, e.g., for daily differential and incremental and weekly full backup jobs, or for migration and replication, and you can set a different retention time for each media pool.
=== {{anchor|S3-compatible}}SEP sesam support for S3-compatible cloud storage  ===
As S3 is an open API standard and AWS Simple Storage Service is a sample implementation of the standard, '''SEP sesam Si3 NG can be used with other S3-compatible cloud implementations'''. The configuration and management of Si3 NG in an S3-compatible cloud implementation is similar to the example shown in this article and must follow the same process and rules provided for using Si3 NG with S3.
<br />Note, however, that other S3-compatible cloud environments have not been validated by SEP and may not work in some cases. You are solely responsible for the use of SEP sesam in a non-S3 compatible cloud environment and agree that SEP shall not be responsible for it.


=={{anchor|configuration}}Configuring Si3 NG S3 store==
=={{anchor|configuration}}Configuring Si3 NG S3 store==
Line 81: Line 87:
<li>The predefined number of channels is already available in the '''Max. channels''' drop-down list. The number of available channels depends on your SEP sesam Server package. For details on licensing, see [[Special:MyLanguage/Licensing|Licensing]].</li>
<li>The predefined number of channels is already available in the '''Max. channels''' drop-down list. The number of available channels depends on your SEP sesam Server package. For details on licensing, see [[Special:MyLanguage/Licensing|Licensing]].</li>
<li>From the ''Device server'' drop-down list, select the '''device server''' for your data store.</li>
<li>From the ''Device server'' drop-down list, select the '''device server''' for your data store.</li>
<li>In the '''Path''' field, '''enter the location''' or use the '''Browse''' button to select a directory on the local disk (as for local storage). This directory is only used to store metadata and temporary files for intermediate storage until the data is uploaded to S3.
<li>In the '''Path''' field, '''enter the location''' or use the '''Browse''' button to select a directory on the local disk (as for local storage). This directory is only used to store metadata and temporary files for intermediate storage until the data is uploaded to S3. Ensure that there is sufficient disk-space available for this local storage, e.g., for 50 TiB in S3 or more, make sure it has 20 GiB free space.  
Click '''OK'''.
Click '''OK'''.
<br>
<br>

Revision as of 10:30, 23 February 2022

Other languages:

Template:Copyright SEP AG en

Draft.png WORK IN PROGRESS
This article is in the initial stage and may be updated, replaced or deleted at any time. It is inappropriate to use this document as reference material as it is a work in progress and should be treated as such.


Docs latest icon.png Welcome to the latest SEP sesam documentation version 5.0.0 Jaglion. For previous documentation version(s), check documentation archive.


Overview

SEP sesam v. 5.0.0 Jaglion has introduced a new generation Si3 data store: Si3 NG. It enables you to back up your data directly to the S3 (Simple Storage Solution) cloud storage and restore the items you want directly from there. After an initial full backup of your virtual and physical environment, you can use any backup level (including differential and incremental backups) to back up only new data to the cloud.

Si3 NG offers significantly improved performance for backup, restore and migration, resulting in improved performance, scaling and resource savings. For a comparison of the two deduplication stores, see the section Comparison of Si3 and Si3 NG below.

Powerful restore

The new Si3 NG can detect duplicate data fragments to streamline the restore process. Use the Web Restore Assistant or GUI Restore Wizard to instantly restore your data from backups. Restoring a single file is easy as you can use the search function to find the desired backup and start the restore process to the original location or a different machine.

Si3 NG uses S3 in the same way as local storage, allowing you to use your S3 cloud storage to securely store and retrieve your business data anytime, anywhere. Data encryption and seeding Si3 deduplication store are not currently supported (see the Si3 and Si3 NG comparison section below).

Multiple media pools

With Si3 NG, you can create multiple media pools to provide scalability and granularity of backup jobs. You can create separate media pools, e.g., for daily differential and incremental and weekly full backup jobs, or for migration and replication, and you can set a different retention time for each media pool.

SEP sesam support for S3-compatible cloud storage

As S3 is an open API standard and AWS Simple Storage Service is a sample implementation of the standard, SEP sesam Si3 NG can be used with other S3-compatible cloud implementations. The configuration and management of Si3 NG in an S3-compatible cloud implementation is similar to the example shown in this article and must follow the same process and rules provided for using Si3 NG with S3.
Note, however, that other S3-compatible cloud environments have not been validated by SEP and may not work in some cases. You are solely responsible for the use of SEP sesam in a non-S3 compatible cloud environment and agree that SEP shall not be responsible for it.

Configuring Si3 NG S3 store

SEP sesam enables you to back up your data directly to S3 cloud storage using Si3 NG deduplication store. The procedure consists of the following steps:

  1. Create a bucket
  2. Create a new backup user
  3. Configure the Si3 NG deduplication store
  4. Configure media pools
  5. Configure backups

Creating a bucket

Sign-up for S3 and create a bucket (container) where every object in S3 is stored. The term bucket is used to describe the container for backup data.

  1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/
  2. Select Create bucket and enter all required information. Follow the Amazon Simple Storage Service instructions: Creating a Bucket.

Creating a new backup user

  1. On S3, configure an AWS identity and IAM (Access Management user <user_name> with administrative user rights. Sign in to the IAM console as the account owner by selecting Root user and providing your AWS account email address.
    Follow the recommendation and use your root user credentials only to create your IAM admin user. For details, see the AWS Identity and Access Management documentation: Creating an Administrator IAM User and Group.
  2. Create the same user <user_name> with the same password on the backup server.
  3. Log in as Administrator with the user type superuser and add the newly created user to the Administrators group. For details, see Configuring Database-Based Authentication.
Information sign.png Note
The S3 credentials are stored encrypted (not in plain text) in the ini file.

Configuring Si3 NG deduplication store

  1. In the Main selection -> Components, click Data Stores to display the data store contents frame.
  2. From the Data Stores menu, select New Data Store. A New Data Store dialog appears.
  3. Under Data store properties, enter a meaningful name for the Si3 NG deduplication store in the Name field, e.g., Si3-NG-S3. Entering the name also creates the name of the drive group for your Si3 deduplication store in the Create new drive group field.
  4. From the Store type drop-down list, select SEP Si3 NG Deduplication Store.
  5. Under Drive parameter, leave the options Create drive and Create second drive checked. The predefined value for the drive is automatically entered in the Drive number field.
  6. By using the additional dedicated drive for the restore, you can perform a backup on the first drive and restore your data from the second drive simultaneously. You can also add a third drive for migration. (For details, see Drive access mode.)
  7. The name in Create new drive group is already created. You can change it by simply entering a new name.
  8. The predefined number of channels is already available in the Max. channels drop-down list. The number of available channels depends on your SEP sesam Server package. For details on licensing, see Licensing.
  9. From the Device server drop-down list, select the device server for your data store.
  10. In the Path field, enter the location or use the Browse button to select a directory on the local disk (as for local storage). This directory is only used to store metadata and temporary files for intermediate storage until the data is uploaded to S3. Ensure that there is sufficient disk-space available for this local storage, e.g., for 50 TiB in S3 or more, make sure it has 20 GiB free space. Click OK.
    If you use the Browse button, the New Data Store information window appears with predefined recommended values for the size of your Si3 NG deduplication store, based on the size of the previously selected local disk (Path).
    Important: Change these values manually under the Size properties, depending on how much storage space you want to use on S3.
    • Capacity: Set the size (in GB/GiB) of the storage for backups.
    • High watermark: The HWM defines the upper value for the used storage space. When this value is reached, the status of a datastore changes from OK to Warning, but backups continue to be performed. Make sure that you provide enough storage space for your backed up data.

    Click OK. You can also change the size of your Si3 NG deduplication store later under Size properties (see Size properties).

    SEP Warning.png Warning
    Since S3 offers unlimited scalability and there is no official limit to the amount of data and number of objects you can store in an S3 bucket, you should set the capacity of Si3 NG on S3 according to your needs. If you know how much capacity you need, you can optimise the cost of your cloud services.
  11. Create Si3-ng-S3.jpg
  12. Switch to the S3 Credentials tab and enter your previously configured credentials, see section Creating a new backup user above.
  13. S3-credentials Si3-ng-S3.jpg
  14. If you want to create a media pool for your Si3 deduplication store immediately, click Create Media Pool. Otherwise, click OK to configure your Si3 deduplication store.

Configuring media pools for Si3 NG

  1. From Main Selection -> Media Pools, click New Media Pool. The New Media Pool window is displayed.
  2. In the Media Pool window, specify the required fields:
  3. Si3-NG media pool.jpg
    • Name: Enter a name for a media pool.
    • Description: Optionally, add a description of the pool.
    • Drive group: From the drop-down list of all available drive groups, select the relevant drive group to which a media pool will be attached. For details on drive groups, see Drives.
    • Retention time [days]: Specify the retention time for the media pool. The retention period begins with the date a saveset is written to the media (at the end time of the first backup) and thus defines the expiry date of the saveset - EOL. When the protection (EOL) expires, SEP sesam can use the media for backups again. For details, see What is retention time and EOL.

You can repeat the procedure and create more media pools for Si3 NG acording to your needs.

SEP Tip.png Tip
  • You can check which media pools are configured with a data store in the data store properties: from Main Selection -> Data Stores -> double-click the selected data store -> select the Media (previously Areas) tab. By clicking the media pool in the list, its properties are displayed.
  • You can also check the Web UI -> Data Stores -> click the Dependencies tab to see the diagram of all dependencies of the datastore and Drives - RDS (first tab) or the datastore and Media Pools - Drive Groups - Drives - RDS (second tab). Hovering the cursor over the diagram shows a tooltip with details for each object.

Si3-ng-S3 dependencies.jpg

Configuring backup

After you have configured an Si3 NG deduplication store and one or more media pools, proceed as follows:

  1. Create a backup task to back up to S3.
  2. Configure a schedule as described in Creating a Schedule.

Once you have configured a task and schedule, create a backup event as follows.

Creating a Backup Event

By creating a backup event, you select the backup level, set event priority and specify where to back up your data to. You can create an event for a specific task or for a task group. The latter enables you to trigger all the tasks in the task group with a single event.

  1. From Main Selection -> Scheduling -> Schedules, right-click the schedule you created earlier and click New Backup Event.
  2. Under the Sequence control, set the Priority of your backup event. SEPuler always executes the schedules with the highest priority first. The default priority level is 1, which is the lowest priority (the highest priority is 99). The only exception are schedules with priority 0, which override all other priorities and are always executed. For details, see Setting Event Priorities. You can also enable the Blocking date. This option should be used together with the high priority for special events. When this option is enabled, events of the same type but with a lower priority are blocked, so that the backup is executed even if other backups are scheduled for the same time.
  3. Under Object, select the task (or task group) you configured earlier and with which you want to link this event.
  4. Under Parameter, specify the Backup level.
  5. From the Media pool drop-down list, select the media pool you created for the Si3 NG deduplication store. The data is backed up to this pool. You can also activate the option SEP Si3 source-side deduplication.
    Si3-ng-S3-bck event.jpg
    • Optionally, specify the drive number of the drive to be used to write the data. Typically, you use this option if you have configured additional drives and want to assign a specific drive exclusively for backup.
  6. In the Follow up field, you can configure events (e.g., migration) that are triggered on the SEP sesam Server as soon as the first event (e.g., backup) is completed. For details, see Follow-up events.
  7. SEP Tip.png Tip
    You can set a follow-up migration task by selecting the previously configured migration task from the Migration task drop-down list.

Monitoring backups

You can view the status of your backup jobs in the GUI (Monitoring -> Last Backup State or Job State -> Backups) or SEP sesam Web UI. The backup status overview provides detailed information about the last run of backup jobs, including the task name, start and stop time of the last backup, backup level, data size, throughput, assigned media pool, etc.

Comparison of Si3 and Si3 NG

SEP sesam v. 5.0.0 Jaglion has introduced a new generation Si3 deduplication store. Si3 offers significantly higher performance for backup, restore and migration, as well as backup to S3 cloud and backup to Azure, the new immutable storage feature SiS, resulting in improved performance, scaling, and resource savings.

Function Si3 Si3 NG
Si3 backup YesY YesY
Si3 deduplication (source-side and target-side) YesY YesY
Si3 replication: local to remote store Notea YesY Si3 V1 to Si3 V1 YesY Si3 V1 to Si3; Si3 to Si3
Si3 replication: to S3 cloud YesY NoN (provides more powerful features for backing up directly to the cloud, see the next two lines)
Backup to S3 Cloud Storage NoN YesY
Backup to Azure Storage NoN YesY (as of Jaglion V2)
SiS (SEP Immutable Storage) NoN YesY (as of Jaglion V2)
Si3 restore YesY YesY
Si3 encryption YesY YesY (as of Jaglion V2)
Seeding Si3 deduplication store Noteb YesY NoN
Usage of tachometer YesY NoN
Notea

SEP sesam does not support a direct upgrade from the Si3 V1 to new Si3. However, to use the new Si3 you can:

  • Back up all data again to the newly configured Si3 deduplication store.
  • After configuring a new Si3, you can also create a replication job to replicate from the Si3 V1 to the Si3 store. Replication reads all the data from the source-side store on the source-side RDS and sends it to the target store using the source-side deduplication function. For details, see Replicating from Si3 V1 to Si3.
  • You can also configure a new Si3 and an old Si3 V1 in parallel on the same host by enabling the key enable_gui_allow_multi_dedup.
Noteb

The Initial Seed feature does not work in v. 5.0.0 Jaglion, but you can use it in earlier SEP sesam versions.