MySource Matrix Resources

Upgrading MySource Matrix from v3.10 to v3.12

Stop Editing

No editing should be taking place during a major version upgrade. Before starting the upgrade, first ensure that all editors are logged out of the system.

Backup

Before starting any upgrade, always backup your MySource Matrix installation. See the MySource Matrix backup management guide for information on how to backup your MySource Matrix system.

Delete Stale HIPO Jobs

During a major version upgrade, stale HIPO jobs should be removed from the system. Log into the MySource Matrix administration interface as root user and go to the HIPO Herder screen. Lock this screen and delete all HIPO jobs that are listed. Note that there may not be any stale HIPO jobs in the system.

Clear the MySource Matrix Cache

There are some changes to the way the MySource Matrix cache is stored. You should clear Matrix's cache from the Cache Manager before starting the upgrade.

Disable Rollback

At this point, you need to ensure that rollback is disabled before continuing. For help with disabling rollback, see the MySource Matrix rollback management guide.

Disable Cron Run script in Crontab

If you use the Cron system, you will have set up an entry in a Crontab file. It is most likely that it would have been set up for the user Apache runs as, however this depends on your system configuration.

As the user the Crontab entry was installed to, run the following command:

$ crontab -e

Find the entry for Matrix, and add a hash ("#") to the front of the entry to comment it out:

# */15 * * * * php /home/websites/mysource_matrix/core/cron/run.php

Get the New Source Code

Now, you need to update the MySource Matrix source code. You can find out how to download the latest stable version of MySource Matrix on the installation page.

Once you have downloaded the source code, you need to replace the existing code with the newer version. This will not reset any configuration options or remove any content from your system.

$ cd /home/websites
$ mv mysource_matrix mysource_matrix_3-10
$ tar -xzvf new_source.tar.gz
$ cd mysource_matrix
$ cp -r /home/websites/mysource_matrix_3-10/data .

Run Install Step 1

The step_01.php installation script adds the new entries for memory limits to the main.inc configuration file.

$ cd /home/websites/mysource_matrix
$ php install/step_01.php .

Set Memory Limits

If you previously edited the files [SYSTEM ROOT]/core/web/index.php or [SYSTEM ROOT]/core/cron/run.php to change the amount of memory used by MySource Matrix's Web or Cron systems from the default 16 MB, you now change them in the main.inc configuration file.

$ vi data/private/conf/main.inc

Find the following lines:

define('SQ_CONF_WEB_MEMORY_LIMIT', 16);
define('SQ_CONF_CRON_MEMORY_LIMIT', 16);

And change the number to the maximum number of megabytes (MB) of memory each system should use. For example, to increase the memory limits for both systems to 40 MB, change the lines to the following:

define('SQ_CONF_WEB_MEMORY_LIMIT', 40);
define('SQ_CONF_CRON_MEMORY_LIMIT', 40);

Note that after this change, you will no longer have to reset your memory limits with each version upgrade, as the settings will be preserved if the upgrade procedure is followed correctly.

Add New Database Fields

There are three new fields that need to be added to the system before running the step_02.php installation script, as indexes are created in the script that rely on these fields existing.

These should be added using the appropriate command line interface for your database (either psql or sqlplus). For example, if your system uses a PostgreSQL database, run the following (this assumes the name of your database is "mysource_matrix"):

$ psql -U postgres mysource_matrix
mysource_matrix=# ALTER TABLE sq_ast_url ADD auth_urlid SMALLINT;
mysource_matrix=# ALTER TABLE sq_rb_ast_url ADD auth_urlid SMALLINT;
mysource_matrix=# ALTER TABLE sq_thes_lnk ADD relid VARCHAR(15);
mysource_matrix=# \q

Reset Rollback

Only run this step if your system has rollback enabled. If your system does not use rollback, you can safely skip this step.

Changes to the above tables also require the rollback triggers to be re-generated, so that all parts of the tables are recorded for rollback purposes. Rollback should be also reset between major versions. For instructions on how to reset rollback, see the MySource Matrix rollback management guide.

Run Install Step 2

The step_02.php installation script adds the new indexes for the tables changed above.

$ php install/step_02.php .

Upgrade Data Directories

The structure of the data/private/assets directory has been modified to get around directory limits on systems on a volume using the Ext2 filesystem (which does not allow more than 32,000 files in a directory, simply speaking). The directory now uses a two-level system, the higher level being a hash of the asset ID, with the lower level folders' names being the asset ID itself, as per previously.

First, run the upgrade_data_dirs.php script like this, to get a report on the size of the upgrade job (this may take a little while on large systems):

$ php scripts/upgrade_data_dirs.php -s . --report-only

Once you are satisfied to continue, run the script like this to start the upgrade. The --show-details parameter is optional but is recommended, as it will show each asset as it is processed:

$ php scripts/upgrade_data_dirs.php -s . --start-upgrade --show-details

Upgrade Thesaurus Database Structure

Run the upgrade_thesaurus_db.php script in the scripts/ directory.

$ php scripts/upgrade_thesaurus_db.php .

This script completes the upgrade of the database structure for the thesaurus system, as well as any existing Thesaurus assets. Due to the database changes, this script is required even if you do not use the Thesaurus asset in your system.

Upgrade Search Package (If Using Oracle Database)

If your system is not running on an Oracle database, or is not using the Search package, you can safely skip this step.

The Search package in version 3.12 of MySource Matrix now uses Oracle's Text package to enhance search functionality. However, this requires the Oracle Text index to be regularly regenerated, otherwise data will not be available to searches within MySource Matrix. This requires the creation of a scheduled job within Oracle.

Firstly, the old index table needs to be deleted and re-created, as the table and indexes are of a different format. This can be done using your normal database user. (This step assumes the database username is "matrix", the database password is "password", and the host name is "db.example.com".)

$ sqlplus matrix/password@db.example.com
SQL> DROP TABLE sq_sch_idx;
SQL> EXIT

Then run the step_02.php installation script to re-create the table:

$ php install/step_02.php .

The remainder of the script needs to be performed as a user with sysdba privileges. Firstly, execute permission to the CTX_DDL object - which contains the function used to resync the Oracle Text index - needs to be given to the database user which Matrix uses:

SQL> GRANT EXECUTE ON CTXSYS.CTX_DDL TO matrix;

Then, a job needs to be created in Oracle's scheduling system to regenerate the index. On Oracle 9i, the script to create the job is as follows:

SQL> DECLARE JobID NUMBER;
  2  BEGIN
  3    SYS.DBMS_JOB.SUBMIT
  4    (
  5      job          => JobID,
  6      what         => 'CTX_DDL.SYNC_INDEX(''sq_sch_idx_value'');',
  7      next_date    => SYSDATE+5/86400,
  8      interval     => 'SYSDATE+30/1440',
  9      no_parse     => TRUE
 10    );
 11  COMMIT;
 12  END;
 13  /

Without modification, this script will regenerate the index every 30 minutes. To change this interval, modify the fractional part of the "interval" setting (specified in fractions of a day - for example, 'SYSDATE+1/24' equates to an hourly regeneration instead. As another example, 'TRUNC(SYSDATE+1) + 4/24' indicates a daily reindex at 4am.)

In Oracle 10g, tasks are added to the Oracle scheduler using a different command to version 9i. Please consult your Oracle documentation for the correct method to add the job to the scheduler.

Complete Upgrade Scripts

The remainder of the upgrade scripts can now be run from the command line as follows:

$ php install/step_03.php .
$ php install/compile_locale.php .

Re-enable Cron Run script in Crontab

You can now re-enable your MySource Matrix system's entry in the Crontab in the reverse manner as you did to disable it: by editing the Crontab file and removing the comment (#) from the front of the entry.

You will also need to add the path to the MySource Matrix system root as a parameter to the script. This was added to resolve a compatibility issue with the Cron script in Solaris.

The line in the Crontab should now read similar to as follows:

# for Linux systems
*/15 * * * * php /home/websites/mysource_matrix/core/cron/run.php /home/websites/mysource_matrix

# for Solaris systems
0,15,30,45 * * * * php /home/websites/mysource_matrix/core/cron/run.php /home/websites/mysource_matrix

The "*/15" format of specifying "every 15 minutes" is an extension to the POSIX standard that may not work on some systems (eg. Solaris). If this format does not work, you need to explicitly specify the minutes (as in the second example above).

Modify Messaging Service Configuration

Two new messages were added to the system in version 3.12 that are recommended to be added to the messaging service's Log to Email Whitelist:

• cron.job.startfail - generated when a Cron Job marked as "long-running" (a new term to describe jobs expected to take longer than the Cron Manager running interval) fails to start because another instance of it is still running
• asset.report.brokenlink - generated by the Links Report and sent to users with Administrator permission to an asset containing a broken link

The above codes can be added to the Log to Email Whitelist by a system administrator from the Messaging Service Configuration screen, inside System Configuration.

Review Date Keywords in RSS Feed Assets

If you use RSS Feed assets (found within the News package), please review date-related keywords within them. Previously, plain date keywords (such as %asset_published_date%) would be forced to follow the correct format for the feed type - either RFC 2822 (example: "Fri,  5 Jan 2006 16:12:50 +1100") for RSS 1.0/2.0 feeds, or ISO 8601 (example: "2006-01-05T05:12:50Z") for ATOM feeds.

The behaviour has been changed so that date keywords behave the same as in other assets, which means they are no longer forced into the correct format. Date keywords should use the proper suffix to ensure they are of the correct format. For example, to get the published date in RFC 2822 format (for use in RSS 1.0/2.0 format feeds), for instance, the keyword "%asset_published_date_rfc2822%" should be used instead. For ATOM feeds, use "%asset_published_date_iso8601%" to return the date in ISO 8601 format.

Review Assets Using LDAP Users

There are now two types of LDAP User assets: LDAP User and LDAP Backend User, which mirror the User and Backend User assets in terms of privileges - ie. standard LDAP User assets cannot access the backend.

LDAP Bridge assets now default to exposing LDAP Backend Users, as they were the same permissions the single LDAP User asset type had before this change. However, if you have set up other assets, such as Asset Listings, to use LDAP Users as an asset type (for example, to list or to search), either the listing or the bridge needs to be changed to return correct functionality.

Firstly, review any LDAP Bridges in your system. If the users inside them do not need backend access, change the User Type setting on the Details screen to "LDAP User" and commit them. Or, if you will not change them, change any assets referencing the "LDAP User" asset type and change them to "LDAP Backend User".

Any changes to an LDAP Bridge's User Type setting will not affect currently logged in users' permissions until they log out (or their session otherwise expires). For example, if you downgrade the User Type from "LDAP Backend User" to "LDAP User", users still logged in from that bridge will stil have backend permissions until they log out.

Reparse Designs

For performance and other improvements and fixes, you should reparse all of the designs in your system, including your Login Design and Password Change Design.

Reindex System (If Using Oracle Database)

If you are not using Oracle as your database or you are not using the Search package, you can skip this step.

To return all index entries to your system after re-creating the search indexing table, re-index your entire system through the Search Manager. This may take a while on large systems.

Upgrade Complete

The upgrade of your MySource Matrix system from version 3.10 to 3.12 is now complete.