MySource Matrix Developer

MySource Matrix Developer Newsletter Issue 97
Friday 20th October 2006

covers versions 3.8.7 stable, 3.10.2 stable, 3.11.0 development

News

No News is Good News

There is no general MySource Matrix developer news this week. However, it should be noted that there is a lot of information in the upgrading section this issue that system administrators would do well to read.

New Features

Future Report Regeneration Cron Job

Added to: 3.11.0 development

A new "Future Report Regeneration" Cron Job has been added to reports. This Cron Job, accessible from any Report asset, can be run at a specified time or repeating on a specified schedule (eg. "every 6 hours").

Note that this change also introduces the concept of 'long-running' Cron Jobs to MySource Matrix. These jobs are expected to last a long time in some instances (this Future Report Regeneration job is one of them), and will not automatically cause the Cron Manager to deadlock if they are still running. Instead, a message will be logged suggesting the interval between each run be increased.

Links Report: Broken Link Notification

Added to: 3.11.0 development

The Link Report asset now has a new "Report Broken Links?" attribute, which will log an internal message to users with Administrator access to the asset if broken links are found in the system.

A new "Report Broken Links Now" button has also been added, which sends these messages based on the current report without re-generating it. Note that this button is only enabled if links are not reported during generation, and once clicked will be disabled until the next generation. This ensures that broken link notifications can only happen once per generation of the report.

Form Submission Keyword Changes

Added to: 3.11.0 development

This addition to Form Submission's keywords allows you to omit the asset ID of the form or section and simply specify the question ID, if the question ID is unique to that form. So instead of %question_name_<assetid>_q1%, you could simply use .

Note that this feature will find the first question with that question number, and therefore is probably not safe for forms that use Form Section assets for grouping and sharing. However it will work fine for simple forms that do not use sections.

Search Manager Improvements for Oracle

Added to: 3.11.0 development

The way searching in Oracle is handled has been changed in the development version, to better make use of the Oracle Text package. This change, which involves separating optimised queries for PostgreSQL and Oracle into separate files that are included on demand, allow advanced features such as improved OR operator handling and the use of bracketed queries.

Note that there are significant upgrade queries that will probably require DB administrator attention, for systems using Oracle. Please see the Upgrading section for more details.

New Global Keywords and Behaviour

Added to: 3.11.0 development

New global keywords have been added to MySource Matrix. The %globals_site_*% keywords will be replaced with values from the current site asset, while the %globals_asset_*% keywords are replaced with values from the current asset.

Also, a design is no longer required in order to use global keyword replacements in a bodycopy. Global keyword replacement will occur regardless of the SQ_REPLACE_MYSOURCE_LEVEL_KEYWORDS flag setting.

Safe Trashing Feature

Added to: 3.11.0 development

This new feature is a protection disables the ability to trash assets (or their children) that are lilve, or have notice links to other assets. This can be used to prevent, for example, broken links to assets in bodycopies. Instead of producing warnings, the trashing is blocked outright.

This feature is disabled by default, and disabling Safe Trash yields the previous behaviour. The enabling of this feature is implemented as a global preference, and therefore is customisable on a user group level. If a user is in multiple groups, the Safe Trash only has to be enabled in one user group for Safe Trash to be active for that user.

Strip Formatting Handles Soft Hyphens

Added to: 3.11.0 development

The Strip Formatting feature of the WYSIWYG now has the option of handling soft (or 'optional') hyphens - expressed in HTML as the entity "&shy;".

Planned Development

Enhanced Backend Searching

To be added to : 3.11.0 development
Estimated start date: already in development
Estimated end date: mid November 2006

Upgrading

3.11.0 Development

Additional queries:

To improve performance on systems using rollback (bug #1802), these indexes should be run on the database of MySource Matrix systems using the rollback feature (on both PostgreSQL and Oracle):

CREATE INDEX sq_rb_ast_effrm ON sq_rb_ast (sq_eff_from);
CREATE INDEX sq_rb_ast_efto ON sq_rb_ast (sq_eff_to);
CREATE INDEX sq_rb_ast_lnk_effrm ON sq_rb_ast_lnk (sq_eff_from);
CREATE INDEX sq_rb_ast_lnk_efto ON sq_rb_ast_lnk (sq_eff_to);
CREATE INDEX sq_rb_shdw_ast_lnk_effrm ON sq_rb_shdw_ast_lnk (sq_eff_from);
CREATE INDEX sq_rb_shdw_ast_lnk_efto ON sq_rb_shdw_ast_lnk (sq_eff_to);
CREATE INDEX sq_rb_ast_lnk_tree_effrm ON sq_rb_ast_lnk_tree (sq_eff_from);
CREATE INDEX sq_rb_ast_lnk_tree_efto ON sq_rb_ast_lnk_tree (sq_eff_to);
CREATE INDEX sq_rb_ast_attr_val_effrm ON sq_rb_ast_attr_val (sq_eff_from);
CREATE INDEX sq_rb_ast_attr_val_efto ON sq_rb_ast_attr_val (sq_eff_to);
CREATE INDEX sq_rb_ast_attr_uniq_val_effrm ON sq_rb_ast_attr_uniq_val (sq_eff_from);
CREATE INDEX sq_rb_ast_attr_uniq_val_efto ON sq_rb_ast_attr_uniq_val (sq_eff_to);
CREATE INDEX sq_rb_ast_url_effrm ON sq_rb_ast_url (sq_eff_from);
CREATE INDEX sq_rb_ast_url_efto ON sq_rb_ast_url (sq_eff_to);
CREATE INDEX sq_rb_ast_path_effrm ON sq_rb_ast_path (sq_eff_from);
CREATE INDEX sq_rb_ast_path_efto ON sq_rb_ast_path (sq_eff_to);
CREATE INDEX sq_rb_ast_lookup_effrm ON sq_rb_ast_lookup (sq_eff_from);
CREATE INDEX sq_rb_ast_lookup_efto ON sq_rb_ast_lookup (sq_eff_to);
CREATE INDEX sq_rb_ast_lookup_value_effrm ON sq_rb_ast_lookup_value (sq_eff_from);
CREATE INDEX sq_rb_ast_lookup_value_efto ON sq_rb_ast_lookup_value (sq_eff_to);
CREATE INDEX sq_rb_ast_lookup_remap_effrm ON sq_rb_ast_lookup_remap (sq_eff_from);
CREATE INDEX sq_rb_ast_lookup_remap_efto ON sq_rb_ast_lookup_remap (sq_eff_to);
CREATE INDEX sq_rb_ast_perm_effrm ON sq_rb_ast_perm (sq_eff_from);
CREATE INDEX sq_rb_ast_perm_efto ON sq_rb_ast_perm (sq_eff_to);
CREATE INDEX sq_rb_ast_edit_access_effrm ON sq_rb_ast_edit_access (sq_eff_from);
CREATE INDEX sq_rb_ast_edit_access_efto ON sq_rb_ast_edit_access (sq_eff_to);
CREATE INDEX sq_rb_ast_mdata_effrm ON sq_rb_ast_mdata (sq_eff_from);
CREATE INDEX sq_rb_ast_mdata_efto ON sq_rb_ast_mdata (sq_eff_to);
CREATE INDEX sq_rb_ast_mdata_val_effrm ON sq_rb_ast_mdata_val (sq_eff_from);
CREATE INDEX sq_rb_ast_mdata_val_efto ON sq_rb_ast_mdata_val (sq_eff_to);
CREATE INDEX sq_rb_ast_mdata_dflt_val_effrm ON sq_rb_ast_mdata_dflt_val (sq_eff_from);
CREATE INDEX sq_rb_ast_mdata_dflt_val_efto ON sq_rb_ast_mdata_dflt_val (sq_eff_to);
CREATE INDEX sq_rb_ast_wflow_effrm ON sq_rb_ast_wflow (sq_eff_from);
CREATE INDEX sq_rb_ast_wflow_efto ON sq_rb_ast_wflow (sq_eff_to);
CREATE INDEX sq_rb_trig_effrm ON sq_rb_trig (sq_eff_from);
CREATE INDEX sq_rb_trig_efto ON sq_rb_trig (sq_eff_to);
CREATE INDEX sq_rb_ast_role_effrm ON sq_rb_ast_role (sq_eff_from);
CREATE INDEX sq_rb_ast_role_efto ON sq_rb_ast_role (sq_eff_to);.

Additional comments: 

APPLIES TO ALL SYSTEMS:

Two new internal messages have been added for the Future Report Regeneration Cron Job and broken links reporting features mentioned in this newsletter:

• cron.job.startfail - for when a long-running job fails to start because another instance of that job is already running.
• asset.report.brokenlink - the notification message for a broken link reported by the new feature of the Links Report asset.

It is recommended that they be added to the "Log to Email Whitelist" in the Messaging Service Configuration (available from the System Configuration section) so admins receive the appropriate emails.

APPLIES TO ORACLE DATABASE WITH SEARCH PACKAGE ONLY:

If you are using the search package and Oracle database, the structure of the search indexing table has been changed. Follow these steps to re-create the index:

• Run the following SQL query to drop the search index table:
  DROP TABLE sq_sch_idx;
• Run the following command line (from the Matrix system root) - the search index table will be re-created:
  $ php install/step_02.php .
• Schedule a job in Oracle to re-sync (update) the new full-text index periodically. This is important; if the index is not updated, new details will not be made available to the search index.
  
  NOTE: the scheduling systems differ in Oracle 9i and 10g. Although sample queries are provided below, both schedulers are rather flexible and the full range of options cannot be shown in this newsletter. A more detailed guide MAY be made available in the future (possibly closer to the release of MySource Matrix 3.12.0), however it's best to consult your local DB administrator or Oracle documentation on advanced options.
  
  In Oracle 9i, this query should create the job. This will set the index to first update 5 seconds from the creation of the job, then every 30 minutes after that. The interval period can be adjusted, keeping in mind the period is expressed in fractions of a day:
  
  BEGIN
  DECLARE JobID NUMBER;
  SYS.DBMS_JOB.SUBMIT
  (
  job => JobID,
  what => 'CTX_DDL.SYNC_INDEX(''sq_sch_idx_value'');',
  next_date => SYSDATE+5/86400,
  interval => 'SYSDATE+30/1440',
  no_parse => TRUE
  );
  COMMIT;
  END;
  
  In Oracle 10g, this query should create the job, although this is untested. Any feedback on an Oracle 10g system would be much appreciated. This should set the index to first update at the specified time, then every 30 minutes. The interval can be changed by changing the "INTERVAL=30" part for a different number of minutes. The interval unit can be changed to, eg, "HOURLY", "DAILY", "SECONDLY", and so on:
  
  DBMS_SCHEDULER.CREATE_JOB (
  job_name => 'SQ_RESYNC_SEARCH_INDEX',
  job_type => 'PLSQL_BLOCK',
  job_action => 'BEGIN CTX_DDL.SYNC_INDEX(''sq_sch_idx_value''); END;',
  start_date => '<first run date here>',
  repeat_interval => 'FREQ=MINUTELY; INTERVAL=30',
  enabled => TRUE,
  comments => 'Updates the Search Index every 30 minutes'
  );
  
  
• Reindex the system in the Search Manager. Remember that the newly re-indexed assets will not be made available to searches until the next run of the job.

3.10.3 Stable

Additional queries:

To improve performance on systems using rollback (bug #1802), the indexes listed in the '3.11.0 Development' heading of this section should be run on the database of all MySource Matrix systems using the rollback feature.

Additional comments: No additional upgrade steps are required.

3.8.8 Stable

Additional queries:

To improve performance on systems using rollback (bug #1802), the indexes listed in the '3.11.0 Development' heading of this section should be run on the database of all MySource Matrix systems using the rollback feature.

Additional comments: No additional upgrade steps are required.

New Bugs Added To Bug Tracker

Bug ID	3.8.x	3.10.x	Dev	Description
1788	N	Y	Y	URLs to non ACSI signs
1789	Y	Y	Y	Remapping URLs does not work when moving to trash
1790	Y	Y	Y	Web paths sometimes not updated when moving assets
1791	N	Y	N	Set Status Trigger Action fails to display backend interface
1795	Y	Y	Y	Backend user can't approve workflow
1797	Y	Y	Y	Date fields not showing for a search in search form
1798	N	Y	N	Submit button keyword on the Default search bodycopy
1799	N	Y	Y	Strucutre Asset Info Report - HIPO going nuts!
1800	N	N	Y	Future Lineage Cron Job not removing notice link when completed
1802	Y	Y	Y	sq_eff_from and sq_eff_to need indexes on rollback tables
1803	N	Y	Y	Some server response break the report in IE6
1806	N	Y	Y	LDAP user email attribute is of type 'text'

Notes:

Bug #1740 (Creating New Paint Layouts and New Divs in Paint Layout Type Formats) was submitted by James Hunter but was marked as Can Not Replicate after the problem was no longer able to be reproduced during debugging.

Bug #1759 (Asset Map Root Node setting not working as expected) was submitted by Avi Miller but was marked as Can Not Replicate after a developer was repeatedly unable to reproduce the problem in both 3.10.2 release and 3.10 branch code.

Bug #1768 (Dynamic Parameters and File Upload) was submitted by kathyc but was marked as Can Not Replicate after a developer was unable to reproduce the problem.

Bug #1769 (Anchor Not Working) was submitted by kathyc but was marked as Can Not Replicate after a developer was repeatedly unable to reproduce the problem.

Bug #1801 (Simple edit access to export form submissions not working) was submitted by Danny Peters but was marked as not being a bug as the layout keywords for the sections had changed in version 3.10.

Bug #1805 (Structured asset report get the HIPO to go over 100%) was submitted by wombat but was a duplicate of bug #1799 (Strucutre Asset Info Report - HIPO going nuts!) submitted by coolvik. The bug has been fixed, and the duplicate has been closed.

New Feature Requests Added To Bug Tracker

Bug ID	Description
1792	GUI for file import script
1793	Allow the strip formatting tool to strip "­"
1794	Reject email notification needs to be sent to the user who started the workflow
1796	Selective Emails screen on the custom form
1804	Add Multiple Calendar Events from Front-End Asset Builder
1807	Keyword: - does not work for chosen asset

Bugs Fixed In The Bug Tracker

Bug ID	3.8.x	3.10.x	Dev	Description
1764	N	Y	Y	Client Side validation for email type questions
1783	N	N	Y	Listing Engine with keyword dynamic root nested into a bodycopy
1786	Y	Y	Y	cannot delete questions from custom form sections from the asset map
1788	N	Y	Y	URLs to non ACSI signs
1789	Y	Y	Y	Remapping URLs does not work when moving to trash
1790	Y	Y	Y	Web paths sometimes not updated when moving assets
1791	N	Y	N	Set Status Trigger Action fails to display backend interface
1795	Y	Y	Y	Backend user can't approve workflow
1797	Y	Y	Y	Date fields not showing for a search in search form
1799	N	Y	Y	Strucutre Asset Info Report - HIPO going nuts!
1800	N	N	Y	Future Lineage Cron Job not removing notice link when completed
1802	Y	Y	Y	sq_eff_from and sq_eff_to need indexes on rollback tables
1803	N	Y	Y	Some server response break the report in IE6
1806	N	Y	Y	LDAP user email attribute is of type 'text'

View Comments