Bob Hoffman
posted this on February 18, 2010 10:58
The short answer is to copy the C:\Program Files\mktdelta\data_f folder (if XP) OR C:\Users\YourName\MarketDelta\data_f (if Vista or 7) . This will copy all your configurations, data, AND registration.
Below is detailed Database information:
Market Delta Database Concepts
Market Delta utilizes a powerful embedded database management system to record and maintain historical data, user preferences, and definitional data. The database contains many different kinds of records, for example there is a record for each instrument. These in turn are related (linked) to the records containing historical data for the instrument. Similarly, relationships are maintained between instruments and the quote pages in which they appear, between instruments and trading positions within portfolios, between instruments and news items and trading notes pertaining to the instrument. Preferences and definitions of charts, quote pages, scans, signals, custom indicators, and all other saved objects are kept in the data base.
The Market Delta database is a collection of 59 files kept within a folder (directory) named data_f inside the Market Delta startup directory/folder. Since many of the individual files are inter-related, the entire collection of files is backed-up and restored as a whole.
Market Delta has an automated daily back-up feature and built-in database recovery system that will restore the database back to its most recent back-up if the database is damaged in any way due to a power failure, system error or application failure. The Backup Preferences window allows the user to specify when daily backups are to be made and the location where the backup is kept. The back-up location is by default the backup_f folder (directory) within the data_f folder (directory). The back-up location may be on another hard drive or some other back-up volume if you like. Set the backup preferences for daily backup at some convenient time of day, or if you typically run Market Delta all day and exit the software regularly each day, setup the preferences to backup when quitting Market Delta.
For new Market Delta users, during initial setup of Market Delta, as instruments, quote pages, charts and portfolios are being defined, it is good practice to periodically issue the Control: Database Utilities: Backup Database command to make a database backup. This assures that all of the setup work up to that point will be protected in the event of a system or application failure. Consider adding a keyboard shortcut that will invoke the Backup Database utility. This can be done easily by creating a "Schedule" (using Setup: Schedule) and assigning a function key, e.g. F2, to the schedule. The schedule will have one "action", i.e. "Backup Database", which is one of the Database Functions you can specify as an action.
When Market Delta starts up, it determines whether the database was closed normally the last time Market Delta ended. If so, startup continues normally. If the database was not closed properly, Market Delta launches the database maintenance utility application to correct minor errors and verify the integrity of the database. If the database is okay, startup continues normally. If the database is damaged, the user is given the opportunity to restore the database to the most recent backup. The database maintenance utility process can take anywhere from a few seconds to several minutes to complete. The time depends on the speed of the computer and hard disk, the number of instruments tracked, and the amount of historical data on file. Options are provided in Backup Preferences for automating the recovery process. It is strongly recommended that the user enable the option to verify the database before making backups and enable the option to automatically restore the database if errors are encountered. If these options are not enabled, Market Delta cannot automatically protect against data loss or loss of the entire database in the event of system or application failure. The reason these protection features are "optional" is to provide flexibility to those users who have exceptionally large databases who wish to take special action in the event of a system or application failure.
Daily backup is necessary. Additionally, a weekly or monthly backup to some other media is advised just in case hard drive failure renders both the database and backup unrecoverable. We recommend at least a monthly backup to CD ROM. Again, in the early phases of setting up Market Delta or when instruments, charts, quote pages, and portfolios are being defined, it is recommended that the database be backed up even more frequently using the Backup Database menu item in the Control menu, under Database Utilities.
Database Integrity
If you experience a system or application failure while Market Delta is running there is a possibility that database integrity may be compromised. Market Delta guards against this by performing frequent "checkpoints" to assure that all database updates are written to the hard drive periodically and every time you "save" an object of some kind, e.g. a quote page. Nevertheless, when a failure occurs, there is a possibility of database damage. Market Delta checks the integrity of the database when the software is started again. If the database does have errors, it is imperative that steps be taken to eliminate the errors before proceeding to use the database. If you checkmark the option "Restore database at startup if verify finds errors", the process is entirely automatic. This is the recommended setting. If Market Delta is setup as recommended to backup daily, to verify before making a backup, and to automatically restore if errors are detected at startup, then there should never be any problem with database integrity.
Further, the MS Windows version of Market Delta incorporates a "crash guard" feature. During execution of Market Delta, if the software encounters a fatal error and must be terminated for any reason, the crash guard feature performs database housekeeping functions to properly close the database files before Market Delta is terminated. This feature protects the database from damage due to application failures.
Rebuilding a New Database
If you find it necessary to rebuild your Market Delta database, here are some tips that will assist you in salvaging as much data as possible from the damaged database.
First, set aside the preference file. On the MS Windows platform the file name is preferen.d01 This file contain registration information and other user preferences. Make a copy of this file and place the copy on the desktop. Be sure to leave the original file in the data_f folder. Check the dbmaint.txt report to be sure that there were no errors reported in either the object.d01 or the custcol.d01 files. If these files show no error, set copies of them aside as well. These files contain chart, RTL objects (scans, signals, custom indicators), and custom column definitions. In summary here are the files you should copy and set aside on your desktop:
Preferen.d01
Object.d01
Custcols.d01
Start Market Delta (i.e. running with the damaged database) and use the File: Export command to export the .All Symbols quote page and each user defined quote page you wish to retain. The exported text files can be imported into the new database later to define the quote pages and instruments.
If database errors were detected in the object.d01 file, you can save all chart and RTL object definitions. Open each chart and RTL object and use File: Export to save their definitions to text files. Repeat this process for each RTL object and chart you wish to retain.
If you wish to save historical data, use the File: Export: Data command. For example to save all daily historical data, choose File: Export: Data, pick the ".All Symbols" quote page, choose Daily, and click the "Export Quote Page" button. A text file containing all daily data on file for every instrument in the ".All Symbols" quote page will be placed into a text file. Tick Data, weekly data, and monthly data may be similarly exported to text files. These files may later be used via the Import command to establish the historical data in the new database.
Having saved quote pages, historical data, charts and RTL objects if necessary, and having set aside the preferences, object, and custcols files as described above, quit Market Delta and install a "fresh" database". To accomplish this you must first remove the data_f folder/directory from the Market Delta folder (remove it or rename it in place if you like) then run the Market Delta "Full Installer"; it will install a new starter database, creating a new data_f folder.
Before starting Market Delta on the freshly installed database, place the "set aside" preferences, object and custcols files into the data_f folder, replacing the three files having the same name. In addition you MUST run the database maintenance function BEFORE starting Market Delta. On the Windows platform run the database maintenance from the "Start" Menu in your Windows taskbar. If the maintenance utility continues to report errors, then contact MarketDelta support for assistance; attach the dbmaint.txt report file to the email.
If the new database (with your old preferences, object and custcols files) verifies as okay, start Market Delta. The application will start up with your registration and other preferences. The charts, scans, trading systems, custom columns will be available in the new database.
The next step is to import all of the instruments. Choose File: Import: Quote page and import the ".All Symbols" quote page you exported earlier. Repeat this same process for any user-defined quote pages exported earlier.
If your object.d01 file was unusable you can recover RTL objects and chart definitions saved earlier using File: Import to import each of them.
Finally, import any exported historical data. Choose File: Import: Data and pick "Daily Historical Data" as the Import Type and "ASCII Text File" as the Import File Type, and "Import Format" as the File Format. Then click the Import button and pick the file of daily data exported earlier. Repeat this process for tick data, weekly or monthly data, if applicable.
At this point you should be up and running with a new Market Delta database. Before proceeding further, make sure the "Verify before Backup" checkbox is marked in your Backup Preferences. Then make a backup of the new database by choosing Backup Database from the Control: Database Utilities menu.
At the present time, there is no provision for transferring portfolio definitions and transactions. Portfolios will have to be created again in the normal fashion. As you continue the setup process, make periodic backups to assure all of your changes are backed up.
Database Troubleshooting
Version Consider this scenario: You employ all of the recommended settings, and have a verified daily backup (made yesterday) and at the end of the trading day, Market Delta attempts to make a new backup and detects database errors. A message appears:
"Database errors were encountered. The requested backup was not performed. Press "Fix" to perform database maintenance, "Restore" to restore the most recent backup."
Market Delta gives you the choice of either attempting the fix the existing database, or simply restoring the most recent backup. Attempting to fix the existing database is certainly worth a try. The "Fix" button runs the "Database Maintenance" utility which can very often fix minor database issues while preserving all of the setup work done since the last backup was made. If you have done little or no setup work or if reverting to the prior day's backup is acceptable, then lick "Restore". If you elect to fix the database, the database maintenance utility will run and inform you whether the database is okay. In the event that the database maintenance (fix) does not repair the database, you have two choices: either (1) restore the previous backup using the Restore Database command in the Control Menu, or (2) continue to troubleshoot the problem.
If you elect to continue troubleshooting, here are some tips. Quit Market Delta; run the database maintenance utility independently of Market Delta. MS Windows users can do this via the Start Menu, i.e.
Start-->Programs-->Market Delta-->Database Maintenance
The Database Maintenance utility will then produce a text file named dbmaint.txt inside the admin directory/folder. You can email this file to MarketDelta support for diagnosis and recommendations, or open the file yourself and inspect the report. In the report you may find lines of text reporting "Problems at record …", for example:
Problems at record 172:
* set SL_SYM_T_TICK(27) error:
invalid last-pointer:
pointer-dba=[1:3952]
* set SL_SYM_T_TICK(27) error:
invalid first-pointer:
pointer-dba=[1:3951]
If you see SL_SYM_T_TICK or SL_SYM_T_BAR, this indicates that the database error relates to intra-day (tick) data in the database. If you see SL_SYM_T_DAT, or SL_SYM_T_WKT, or SL_SYM_T_MOT, the error relates to daily, weekly, or monthly historical data. If all of the errors relate to either intra-day, or daily/weekly/monthly historical data, Market Delta has initialization utilities that can repair the database while preserving all other data. The Initialize D/W/M Database utility will eliminate all daily, weekly, and monthly data from the database and repair any damaged linkages relating to the storage of daily, weekly, or monthly data for all ticker symbols. Before using this utility, you may wish to use the Export command to Export Daily (or weekly or monthly) data for all symbols to a text file. After initializing the D/W/M database, you can then use the Import command to bring the exported data back in. Similarly, the Initialize Intra-day Database utility will eliminate all intra-day historical data from the database and repair the linkages. Export one minute bar data for all symbols before initializing if you like; alternatively, you can download intra-day data as needed if your data service supports a historical intra-day server.
After initializing and re-importing historical data, run the database maintenance utility again to assure that the database now verifies okay.
Another type of database error you may see in the dbmaint.txt report involves the linkage SL_SYM_T_NEWS. Problems of this kind indicate an error in the news portion of the database. If you do not wish to restore your backup, you can eliminate the problem by using the "Initialize News Database" utility. All news items will be discarded from the database and all linkages will be repaired. After initializing the news database, run the database maintenance utility again to assure that the database now verifies okay.
Another type of database error you may see in the dbmaint.txt report involves the linkage SL_SYM_T_NOTE. Problems of this kind indicate an error in the trading notes portion of the database. If you do not wish to restore your backup, you can eliminate the problem by using the "Initialize Trading Notes" utility. All trading notes will be discarded from the database and all linkages will be repaired. After initializing the trading notes database, run the database maintenance utility again to assure that the database now verifies okay.
If errors persist, contact MarketDelta support for further assistance and guidance.
More Database Details
All software products that provide charting must have some place to store the data. Metastock, for example, a widely used charting program, stores each stock's data in a file on the hard drive. If you track 1000 stocks you have 1000 historical files, each one containing the historical data for one instrument. Market Delta on the other hand uses an integrated database management system to record all information about the stocks you are following and the historical data for each one. Market Delta includes an embedded database system that manages all of this information. For example, Daily/Weekly/Monthly historical data for all instruments whether you track 10, 100, 1000, or 10,000 ticker symbols is kept in a single file. The database software embedded in MARKET DELTA keeps track of which history records belong to which instrument. Another file keeps track of each instrument. There are linkages between this file (the symbol file) and the history file. The record for IBM in the symbol file contains information that tells the database system where in the history file to find the historical data for IBM. This is what I mean by "the files are linked or interrelated".
There are many reasons for using an embedded database system rather than a more simplistic "one file per" approach. When MARKET DELTA does a scan on 1000 stocks it only has to open the historical file one time instead of opening and closing 1000 individual files. Scans run very, very quickly in MARKET DELTA. Another reason is that a database system is capable of managing very complex data relationships easily. The MARKET DELTA database keeps track of instruments, quote pages, charts and their contents, layouts, scans, custom indicators, trading signals and trading systems, and much more. Most of these things are interrelated and the database system allows MARKET DELTA to quickly determine, for example, which tickers belong in which quote pages, which portfolios contain positions on a particular stock, which indicators and instruments belong in each named chart, which instruments make up a custom instrument or even which custom instruments are used in other custom instruments.
We use the database system to great advantage in MARKET DELTA to record all of your preferences and definitions and all of the complex relationships among them. The flip side is that the care and maintenance of this small collection of inter related files is very important. That's why we employ daily verification and backup methods and have a wide variety of database utility functions aimed at maintaining the integrity of the "database".