Clone prod environment to dev (or other)

Ever wondered how you can clone your productive environment to your development environment to work, test and develop with real data?

That’s what we want in our company. For now we made all changes in the productive environment, as the initial configuration. But after the go live, we wanted to test the changes and planned improvements for the ServiceNow experience before we implement those in our productive instance.

Note: The cloning process in ServiceNow is handled as a change request in the HI portal.

Clone process

In the following image you’ll find the clone process.

Clone process overview

The following tasks will be performed of the ServiceNow platform:

  1. Generates a file to preserve operational data on the target server. This file contains the data preserved by data preservers.
  2. Copies the database schema from the source instance to the target instance.
  3. Create tables in the target instance database using the source instance table definitions.
  4. Copies data from the most recent nightly backup of the source instance to the target instance database.
    Certainly large tables are normally excluded. These include audit, log, and email tables.
  5. Briefly disables UI traffic and requests to the target instance server.
  6. Displays the message Clone in progress… to any user accessing the target instance.
  7. Restores operational data preserved from the target instance.
  8. Runs any post-clone cleanup scripts on the target instance.
  9. Briefly suspends all email functions on the target instance.
  10. Queues any event to regenerate text indexes.
  11. Enables UI traffic and requests to the target instance server.

What you’ll need:

Roles: Clone_admin
URL of target instance
Admin username and password of target instance

Note: The target cannot be a live production instance

Note: Ensure the target instance, whether dev, test, stage or other, has the system property glide.db.clone.allow_clone_target enabled.

After you provided all information, we will prepare the cloning process.

Source instance:

Co create a target, go to the module Administration -> Clone Targets and hit new.

PJ0•essed * 
加 0 ) e " eu ! 山 e 
山 e 山 1 * 
u 丨 ~ 1 d 川 uno a 川 艹 0 | ) 01 
Ⅵ nb , , 一 吅 •p 亅 p 」 0 」 Mau

Select the target instance

Now we created a target entry, we will request the clone process. Therefore navigate to System Clone -> Request Clone.

You are asked to provide the target instance. Therefore select the previous created instance.

In the next field, you can define a schedules start time for the clone process.

Now a form is displayed where you have to set the necessary information:

Field Description
Target instance Select the previous created instance as target.
Clone Schedules Start time Note: I’d like to recommend to clone the instance outside the business hours due to performance issues of the productive, source instance.
Email upon completion Email recipient when the clone is done or failed

Clone options

Field Description
Exclude tables specified in Exclusion List Prevent cloning records from the source instance specified in the System Clone > Exclude Tables module. Use this option to create empty but usable tables on the target instance. By default, the system excludes tables for auditing, license usage, logging, and notifications. This option is selected by default. Note: This option is not supported by the legacy clone engine.
Exclude audit and log data Prevents cloning audit and log records from the source instance. Use this option to create empty but usable audit and log tables on the target instance. This option is selected by default.
Exclude large attachment data Prevents the cloning of large attachments such as video files, image files, and other typically large binary file types. Excludes all common binary file types, regardless of file size. When selected, the clone also excludes attachments from the Attachments [sys_attachment] and Attachment Documents [sys_attachment_doc] tables that meet all these criteria.
The attachment table_name value does not indicate it is a small file. Small attachments have table name values that start with ZZ_.
The attachment data type value indicates it is a large file such as application or video.
The attachment table_name is not one of these system tables: sys_certificate, ecc_agent_jar, ecc_agent_mib, sys_store_app, or invisible.sys_store_app.
Amount of data copied from large tables Select the number of days of historical data to clone from large source tables, including task, pa_scores, and metric_instance. By default, the target instance receives the latest 90 days of historical data from the source instance.
Preserve theme Preserves the theme and CSS elements on the target clone.
Preserve users and related tables Prevents Users and user-related tables from being overwritten on the target clone.
Clone frequency* Specify how often this target automatically receives clone data. For more information about scheduling cloning, see Schedule cloning.
No. of occurrences* Specify the number of automatic clonings. The maximum value is 9.

*This is not necessary in our use-case.

System Clone 
Subrnit 
Cloning copies configuration and application data from the source instance to the target instance (More info). Backup 
any unpublished custom applications and update sets tkfore requesting a clone (More info) 
Cbræ 
Chne 
* Target 
* Clone Scheduled Start Time 
Email upon completion 
Options 
Exclude tables 
Exclude 
Q 
Amount of data tables 
Last 90 days 
and tables 
O

After you provided all information and hit the submit button, you are asked to authenticate with the credentials of the target instance admin.

Machine generated alternative text:
Authenticate Target - 
To clone to an instance, ycnl must have admin privileges on the target instance 
Instance URL: https:/,a........dev.service-now.com 
Username: admin 
Password: 
Authenticate
Machine generated alternative text:
Confirm 
Warning! 
Any update sets, development work, and data in the target instance will be 
overwritten. 
Do you want to proceed? 
Cancel

When everything is prepared, you’ll receive change request notifications for your clone request. Those notifications are sent from the HI portal.

I’d like to show you this notifications, but there are sensitive data contained, so you’ll need to view those on your own. 😉

Cancel a clone

Required role:
Admin

In some cases you have to cancel a clone process for a reason. To cancel the clone, you simply go to the module System Clone -> Live Clones -> Active Clones. You’ll find a list with the active clones. Now open the wanted clone. The option to cancel a clone, you find on the Related Links section. Be careful, the clone process is directly canceled. You won’t receive any confirmation page.

Cbne 
= System Clone I clone_request view•l 
Clonng copies configuration and application data from the source instance to the target instance (Yore info). Backup 
any unpu%hed custom applications and update sets before requesting a clone (More info) 
* Target in 
* Clone Scheduled Start Tirne 
Email upon completion 
Options 
Related Links 
Cancel Clone

Duration and completion

The duration of the progress depends on your amount of data and which tables you defined for the cloning.

For comparison, we have productive data from two months and not excluded any table. In our case, the cloning took about 00:45h for completion. You can see the history as well as the Clone Logs in the System Clone -> Live Clones -> Clone History module.

Rollback

After clone completion, you have up to contact ServiceNow Customer Support and request a rollback of the target instance to its pre-clone state. You are notified when the rollback is complete.

Referred Links

https://docs.servicenow.com/bundle/newyork-platform-administration/page/administer/managing-data/concept/c_SystemClone.html

Save the date… Now at Work – Frankfurt am Main: 30. Oktober 2019

Now at Work – Frankfurt 2019 – Ich bin dabei!

Es ist eine meiner ersten Konferenzen und die erste von ServiceNow überhaupt. Ich bin unglaublich froh, dabei zu sein!

Die Konferenz findet am 30. Okober 2019 im Congress Center Messe Frankfurt statt.

Über Now at Work – Für eine Welt, in der Arbeit weniger Arbeit macht

Now at Work bringt außergewöhnliche Menschen zusammen, die die Arbeitswelt verändern. Besuchen Sie uns in Frankfurt und erleben Sie, wie digitale Workflows die Arbeit für jeden sinnvoller und produktiver machen. Für eine Welt, in der Arbeit weniger Arbeit macht.

Lassen Sie sich inspirieren

Experten aus renommierten deutschen Unternehmen teilen mit Ihnen aus erster Hand, wie vormals fragmentierte Prozesse in ihren Unternehmen durch intelligente, digitale Workflows transformiert wurden.

Informieren Sie sich

Es erwarten Sie spannende Vorträge und praxisnahe Trainings mit technischen Experten und Entscheidungsträgern, die Sie dabei unterstützen, leistungsstarke digitale Workflows auch in Ihrem Unternehmen zu entwerfen und zu implementieren.

Vernetzen Sie sich

Tauschen Sie sich aus mit ServiceNow-Kunden, -Partnern und -Experten aus Deutschland, die Sie bei Ihrer digitalen Transformation inspirieren und unterstützen können.

Werden Sie aktiv

Erstellen Sie neue Enterprise Workflow Apps, oder erleben Sie einen Deep Dive zu den neuen Funktionen der Now Platform.

Hier findet Ihr das Programm & Agenda.

Schedules Public Holiday Germany for SLA Definitions

In the company I’m employed at we’re currently planning our service level agreements based on the different contracts, wondering how we can do justice to the situation of various public holidays of the German states.

Now I’m sitting here and wondering, how to easily create a single data source in excel for the public holidays in Germany and import them all at once.  There should be a solution to make a bulk import but I’m not able to find something on the world wide web.

So now I’m on my own. To my luck there are some default schedules for the U.S. Holidays on which I can orient myself.

The entries shown there are very well. But what’s with the dynamic holidays like Easter Sunday? – One solution came around in mind: Create an own schedule with all dynamic days for this and then create an parent-child relationship. For this it should be possible to create an excel file and then import everything all at once.

Export source data

First of all, I viewed the schedules table (cmn_schedule) where all schedules are stored. To export this table, you only need to click on one column option , select Export and then Excel (.xlsx).

To export the second table, I had to find the name. For this, I opened an schedule entries entry and right clicked on Show – ‘name’. After this, a small window appeard and I was able to see the table name (cmn_schedule_span):

I searched for this table by typing in cmn_schedule_span.list in the navigation field and hit enter. I exported this table like the one before.

Now I have the data of both, the schedules and the schedule entries.

From now it would be easy – I thought.

After I viewed the Content frame, I saw a related list which handles the parent/child relationships. This is a requirement as well. To find this table, I had to search a lot. But I’ve done it.

You need to klick in the table, e.g. Child Schedules on the Column Options , Configure and then select table. A new form is shown and you can see the table name cmn_other_schedule. Now I searched again in the navigation field to view the entries and export them.

Now I have all three source tables where I can orient myself:

Cmn_schedule.xlsx
– all schedules are stored here

Cmn_schedule_span.xlsx
– all schedule entries are stored, like the holiday “Neujahr” and it’s value, even if and when it’s repeated.

Cmn_other_schedule.xlsx
– all parent child relationships are stored here

Plan the data

I have opted for a detailed and yet simple method. For each object I make an own schedule where either the entry is stored or at least the reference to the child schedule.

So I had to gather all the information. You can see them in the following list:

  • Allerheiligen
  • Augsburger Friedensfest
  • Baden-Württemberg
  • Bayern
  • Berlin
  • Brandenburg
  • Bremen
  • Buß- und Bettag
  • Christi Himmelfahrt
  • Deutsche einheitliche Feiertage
  • erster Weihnachtstag
  • Fronleichnam
  • Hamburg
  • Heiligabend
  • Heilige drei Könige
  • Hessen
  • internationaler Frauentag
  • Karfreitag
  • Mariä Himmelfahrt
  • Mecklenburg Vorpommern
  • Neujahr
  • Niedersachsen
  • Nordrhein-Westfalen
  • Ostermontag
  • Ostersonntag
  • Pfingstmontag
  • Pfingstsonntag
  • Reformationstag
  • Rheinland-Pfalz
  • Saarland
  • Sachsen
  • Sachsen-Anhalt
  • Schleswig-Holstein
  • Silvester
  • Tag der Arbeit
  • Tag der deutschen Einheit
  • Thüringen
  • Weltkindertag
  • zweiter Weihnachtstag

In the second step I need the relationship of the individual schedules.

Parent Schedule Child schedule
Baden-Württemberg Heilige drei Könige
Baden-Württemberg Fronleichnam
Baden-Württemberg Allerheiligen
Bayern Mariä Himmelfahrt
Bayern Allerheiligen
Bayern Heilige drei Könige
Bayern Fronleichnam
Berlin internationaler Frauentag
Brandenburg Reformationstag
Bremen Reformationstag
DE consostent Holidays DE consostent dynamic Holidays
Deutsche einheitliche Feiertage Ostermontag
Deutsche einheitliche Feiertage Pfingstmontag
Deutsche einheitliche Feiertage zweiter Weihnachtstag
Deutsche einheitliche Feiertage Tag der Arbeit
Deutsche einheitliche Feiertage Tag der deutschen Einheit
Deutsche einheitliche Feiertage Silvester
Deutsche einheitliche Feiertage Karfreitag
Deutsche einheitliche Feiertage Christi Himmelfahrt
Deutsche einheitliche Feiertage Heiligabend
Deutsche einheitliche Feiertage Ostersonntag
Deutsche einheitliche Feiertage Pfingstsonntag
Deutsche einheitliche Feiertage erster Weihnachtstag
Deutsche einheitliche Feiertage Neujahr
Hamburg Reformationstag
Hessen Fronleichnam
Mecklenburg Vorpommern Reformationstag
Niedersachsen Reformationstag
Nordrhein-Westfalen Fronleichnam
Nordrhein-Westfalen Allerheiligen
Rheinland-Pfalz Fronleichnam
Rheinland-Pfalz Allerheiligen
Saarland Mariä Himmelfahrt
Saarland Fronleichnam
Saarland Allerheiligen
Sachsen Buß- und Bettag
Sachsen Reformationstag
Sachsen-Anhalt Heilige drei Könige
Sachsen-Anhalt Reformationstag
Schleswig-Holstein Reformationstag
Thüringen Reformationstag

To plan the data for the entries, I searched the web to find a list where every holiday in germany and the corresponding states are stored. After a short time I found what I was looking for. On the web page of excelformeln (http://www.excelformeln.de/formeln.html?welcher=3) is a well explained forumla to identify the Ostersonntag. Based on this holiday, it’s easy to calculate each other dynamic holiday. There is also a file stored which already provides every holiday in each state. A nice benefit!

With this file it was very easy for me to plan the holidays and transform them into the file.

The most important fields for the entries are the following in a specific format:

Field name Example
Name Neujahr
All day TRUE
Repeat on 1
Start date time 01.01.2019 00:00:00
End date time 01.01.2019 23:59:59
Class Schedule Entry
Month 1
Monthly type Day of the month
Repeat every 1
Repeat until 0
Repeats Yearly
Schedule Deutsche einheitliche Feiertage
Show as Busy
Type Busy
Yearly type Day of the year

Import the update sets

Warning: it’s necesarry to import the files in the following order. If not, the importing process will  go wrong:

Cmn_schedule

Cmn_schedule_span

Cmn_other_schedule

To import the updates, I first have to load the data. Go to System Import Sets -> Load Data.

I created a new table like displayed in the image and hit submit:

The data was successfully loaded. Now I have to create a transform map. Therefore click on the Next steps Create transform map. After I provided all necessary information, I clicked on Auto Map Matching Fields.

Sometimes you have to use the Mapping Assist to map the source field to the correct target field. After everything has been mapped, you can Transform.

Now you can view the Schedules list and validate the import.

Now repeat this with the other two tables:

Cmn_schedule_span.xlsx
Load Data:
– Import set table: Create table
– Label: cmn_schedule_span holiday import
– Source of the import: File
– File: cmn_schedule_span.xlsx
Create transform map:
– Name: cmn_schedule_span holiday import
– Target table: Schedule Entry [cmn_schedule_span]
Auto Map Matching Fields
Mapping Assist (optional) NOTE: check Name field
Transform

Cmn_other_schedule.xlsx
Load Data:
– Import set table: Create table
– Label: cmn_other_schedule holiday import
– Source of the import: File
– File: cmn_other_schedule.xlsx
Create transform map:
– Name: cmn_other_schedule holiday import
– Target table: Other Schedule [cmn_other_schedule]
Auto Map Matching Fields
Mapping Assist (optional)
Transform

Please validate the new Schedules. Now you and we are able to create SLAs to provide correct functionality based on your needs.

I don’t want to deprived the files:

Download