Introduction:
Data migration is a tedious process when comes to moving data between two Jira instances.
Most of the time it gets harder when there is data already exists in the target instance where site restore won’t really help.
So, we have to go with the project import tool to import the projects from source Jira instance that comes with a list of limitations. After several migration experiences, we came up with a list of things to be considered and prepared before starting the migration of data between Jira’s
The following document will walk you through the pre-requisites and steps to be followed to complete an error-free and successful data migration.
High-Level Overview
| Pre- Migration | Migration | Post Migration | ||
| Analyze | Plan | Preparation | Project Import | Validate the migrated project |
| Jira Version Analysis | Project Grouping | Spin up Temporary instance | Import the project | Validate workflow transitions, post-functions, and validators |
| Plugin Analysis | Batching Migration | Creation of empty projects | Resolve Validation Errors | Validate access permissions |
| Shared project configurations | Time Estimation | Migrating Workflows | Complete import process | Validate notifications |
| Number of projects | Resource Estimation | Configuring Projects | Validate agile board setup | |
| Prep the database | ||||
| Backing up data | ||||
| Pushing backups to destination Jira | ||||
| Pushing attachments to destination Jira | ||||
| Create Agile boards | ||||
Plan the migration
Planning the migration will give the team clarity of what to do when to do and who will do it. Also, it saves time when dealing with a large set of data.
Group Projects
Most of the time projects share workflows and other project related schemes such as issue types, screens, and fields. Grouping the projects based on workflows could help admins migrate a list of projects as a batch that shares the same set of workflows. Because workflow issues are difficult to deal during importing since it involves a lot of manual work such as status/transition ids, names, field screen ids, custom field id conflicts that need to be edited to match up with the ids in the target Jira instance rather than dealing with any other issues such as custom fields and custom field options, etc.
We will talk about how to export, edit and import workflows from source Jira to target Jira later in this document.
Temporary Instance and Database
In case of data migration between lower version to higher version, it is always a best practice to site restore the source data into a temporary instance and upgrade the temp instance to the same version as the destination Jira instance to avoid unexpected errors and blockers during migration.
Make sure all the user-installed plugins are of the same version that of the destination instance user-installed plugins to avoid plugin related issues while project import.
Once the upgrade is completed backup the site. This will be the backup we will use for the project import in the destination Jira instance.
Setting up a dummy project for Migration
Since we are migrating the data from one Jira instance to another Jira instance using the import project tool, it is required to create an empty project with the same name and project key with the following setups in place. Note that the project key should be unique. If the key already exists in the destination instance, the source key should be changed or a new
- Issue type scheme
- Workflow scheme
- Issue Type Screen scheme
- Workflow Screens
- Field Configuration Scheme
- Users and Roles
- Permission Scheme
- Notification Scheme
- Agile board and board configurations such as columns, swimlanes, quick filters, Board’s filter query, etc.
For a project that is to be migrated, it is necessary to create a dummy project with all its schema associations configured for the import tool to successfully import issues and its related data into the project.
When we create the screen schemes and workflow screens, we most probably create all the necessary custom fields that are used in the project which will reduce the number of validation errors during project import.
Creating Issue type scheme
For the project to be migrated, list down all the issue types and make sure they already exist in the destination Jira instance. If any of them is not present, go ahead and create one in administration > issues > issue types > create issue types. Issue type names are case sensitive and make sure you leave no extra spaces when creating new ones
Workflow scheme
The difficult one among the above schema configurations is workflows especially when the workflow is complex and includes validations, conditions, third party plugin post functions, etc.
So, we broke it down into simple steps on how to export a complex workflow from one jira instance and import it into another Jira instance.
-
- Export the workflow as XML
- Open the XML using text editing tools such as sublime text or notepad ++ and choose syntax of XML for easy identification of sections definitions
- Status Id Replacement – Find the workflow steps and replace the ids with the id of the
same step in destination workflow. For example, if the status id of step “Approval Requested” in source instance is 15732
Go to destination instance, log in as administrator. Go to Administration > Issues > Statuses.Find the Status “Approval Requested”. Hover on the edit button, you will see the id of the status at the taskbar or click edit and copy the id from the URL.If the Id in the destination is 11533, go back to the text editor replace 15732 with 11533. Repeat it for every workflow stepsIn case if the status is not found in the destination instance, please create one.
- Custom Field Id Replacement – Custom fields are used in workflow post functions which undergoes some actions based on the workflow transitions. You can find those custom field ids in the XML back up of the workflow. To replace those ids with the destination custom field ids, first, we need to find the name, options if it is a list field and its existence in the destination.
- To identify the custom field, Go to source instance > administration > Issues > Custom fields
- Pick a random custom field and click edit. This will let us know what is the custom field id in the URL. Now replace the id with the id copied from the XML and hit enter. Now we know the name and type of the custom fields.
- Create a replica of the field in the destination Jira instance. Copy the id of the field and replace it in the XML file Wherever the field is referenced.
- Resolution Id Replacement -Workflow post functions may include the setting of resolutions when transitioning issues. Find the resolution name of the ids in the source Jira instance by login as administrator. Go to Administration > Issues > Resolutions. By Hovering over the edit button of the resolution, we can find the resolution name of the Id referenced in the XML file. Go to the Destination instance, Find the name of the resolution and note down the resolution’s id value. Then replace the Id with the Id in the XML file. Create one if it is not present in the destination Jira.
- Workflow Screen id Replacement – Similarly we have to make sure the workflow transition screens are created with the set of fields in the destination Jira instance and the ids are replaced in the XML file. If the screens with the fields already exist in the destination Jira instance, we can reuse the ids of those screens.
- Import the Workflow – After all the necessary ids are replaced, save the XML file. Login destination Jira as administrator. Go to Administration > Issues > Workflows. Click Import Workflow and select XML import. Copy and paste the XML code into the editor. Enter workflow name and click import. After the import, edit the workflow in diagram mode and arrange the steps and transition arrows the way it is readable to anyone who looks at the workflow.
- Caution! – > Sometimes the post functions and validators in the create issue transition can cause errors while importing issues due to lack of create issue permissions or validators and post functions that does not satisfy when importing issues. For example, validators, such as users with the role and required field validators may cause import failures. To avoid such errors, make a note of those validators and post functions and remove it from the create issue transition. You can add it back once the project import is completed.
Issue Type Screen scheme
This is a three-step process where you create screens with all the fields required for the issues in the selected project and a screen configuration that defines screens for issue operations such as create, edit and view. This screen scheme will be associated with the issue type screen scheme which defines screen schemes based on the issue types used in the project.
Creating this issue type screen scheme prior to the project migration eradicates project import validation error such as issue type associations with the issues.
Field Configuration Scheme
Replicate field configurations from source project to destination dummy project to configure issue type field requirements to avoid post-migration validation issues.
Users and Roles
Set up project roles for the users that are part of the project, so that while validating the migrated project, users will not face any access-related issues. Create users in case the user is not present the destination instance. Create necessary roles in the project roles section of administration page in Jira
Permission Schemes
Set up the permission schemes which enable necessary permissions for the users involving in the project when validating or working on the migrated project. The right set of permissions avoids issues such as transition errors, sprint management issues, etc.
Notification schemes
Replicate the notification schemes. Sometimes projects use custom notification schemes. Replicating the notification will enable the team to experience a clean project migration. Also, the right team members will receive the right notifications.
Refer here for the project configuration documentation as per Atlassian
Possible Database Errors
There are some errors that block the import or import with errors and missing data which should be overcome by making some database changes. It is better to do it beforehand in the Temporary instance database. Because when we face such errors, we have to make the changes in the temporary instance, then redo the backup, push the backup into the destination instance folders, which cost a lot of time and double work. Below are some of the possible Database errors that could cause import failures.
- Null Project Role – Backup files may contain data with missing project roles where the project role will be NULL. To fix this, run the below database queries before taking the site back up.
Caution! – It is always advisable to have a copy backup of both source and destination instances for safe migration.
SELECT * FROM projectroleactor WHERE projectroleid IS null;
DELETE FROM projectroleactor WHERE projectroleid IS null
- No ‘issue’ field for CustomFieldValue xxxx – There may be some custom field values without an issue filed id mapped. To fix this run the below database queries
SELECT * FROM customfieldvalue WHERE issue IS NULL;
DELETE FROM customfieldvalue WHERE issue IS NULL;
- Epic Color issue– When migrating data from versions 5.x to 6.x and later, you will face epic color name issues. Since this field is a locked field in Jira, to rename the field, the field should be unlocked. Run the below database queries to unlock the field to edit. Kindly note down the id of the epic color field. It will be something like “customfield_10505”
SELECT id,customfieldtypekey,cfname from customfield where customfieldtypekey like ‘com.pyxis.greenhopper.jira%’;
SELECT * from managedconfigurationitem;
UPDATE managedconfigurationitem set managed=’false’ where item_id in (‘customfield_10505’)
Moving the backup from Source Jira to destination Jira
Backup the temporary instance and download the.XML back up file. Note that this back up does not include the attachments
Push the backup file into the import folder in the home directory of the destination Jira instance.
Attachments are stored in <jira-home>\data\attachments directory. Move it to <jira-home>\import\attachments. Make sure the attachment folder has the necessary read/write permissions enabled. Otherwise, the import fails with there are attachments that cannot be imported.
Issues that require editing the entities.xml file
“There are ‘1’ required user(s) that Jira cannot automatically create” – This is due to a required user(s) where there is no real user exists but refers to a null value in the XML file. For a Linux/Unix based system run the below command.
Log in to the system and go to the backup file directory/home/import
Unzip the file and edit the entities.xml file to remove all occurrences of assignee=””. Note: If you’re using an Unix-based operating system, you can run the below command on a terminal window.
$ sed -i ‘s/assignee=””//g’ entities.xml
Zip the backup back together before rerun the project import with the fixed backup.
That’s it! Now we have everything set up for Project import.
Importing project from the backup
Login as administrator in the destination Jira instance. Go to System > Project import
Enter the file name of the backup. For example Jirabackup.zip and click next.
Choose the dummy project created. Click import. Review the validation errors. At this level, we have already cleared some of the issues during the prerequisites above which causes redoing and time-consuming. Now if there is any validations error such as custom field name, user group creation, custom field options, etc. can be fixed within the destination instance and proceed to import.
Once the project import is success full, validate the number of issues, attachments, components, and versions created.
Agile Board setup – If the migrated project has Scrum or Kanban boards, go ahead and create those boards and its configurations. Even though the backup contains the agile data, it will not contain the board setup details. Once you create the boards and its configs like columns, swimlanes, and query filters, the data will be then mapped from the backup.
Final Step – Rebuild index
After migrating the projects, rebuild the index. Run background re-index in case if you want the instance to be running. This may affect system performance until the indexing is complete. Or lockdown Jira and rebuild the index. This will be comparatively quick but the instance cannot be accessed while re-indexing.