The Content Sync Module of Director (CSM) can then be invoked and configured to create a list of HTTP or CIFS objects and folders for pre-population. After the intial scan, CSM can be used to rescan for updates to the content. Once the content list is created, it can be uploaded, (either by schedule or manually), to the Director. Director then pushes the list to each ProxySG within the Content Distribution list.
Content Delivery Network Architecture provides the ability to distribute content across the enterprise network using the Job Scheduler, eliminating the need to backhaul traffic and conserving valuable network resources. Content can be distributed, prioritized and scheduled according to business needs.
Installation requires a Windows or LINUX workstation on which to install the Content Sync Module (CSM) component. Managing the Director appliance requires a Java applet, downloadable from the Director appliance, upon login. The CSM operates by crawling a CIFS, or HTTP server and tracks the time that the content was last modified, pushing the content to the ProxySG appliances accordingly. It will generate a list of files along with their last-modified times and keep that in a flat file rather than a true database. You can then push this content out to your SG network, via Director. These appliances will get that list, delete the objects that shouldn’t be there anymore and download the other files.
Note: The Content Sync Module does not ship with Director, but is available as a separately downloable module. This article assumes you have a working DIrector appliance, and are able to log into it using the Director Management Console (DMC) Java application. For details on how to install the CSM, see 000010802
To create a job that crawls a website for content, follow these steps:
- Invoke CSM on your workstation and click on File and select New Job- you can give it any name you want- we chose the default name, as you can see below.
- Click Schedule box, select Recurring tab and specify Day of the Week and Time.
- Click Add > and confirm your schedule entered into the Result Schedule: (Alternatively you could select One Time tab for scheduling purposes.)
- Under the New Job just created, select Scan > select radio button Scan Directories > under Directories box click Add and select the folder you wish to scan and click Select Directory.
- Once you have added a list of Directories you can individually Enable/Disable via Enable Directory check box..
- Select Upload under Jobs and check Enable upload. Select Upload Type.
- Select Blue Coat Director under Jobs and enter Director IP address and administrator credentials.
- You can specify the protocol connection method as either Telnet or SSHv2 ( The CSM software will use the Expect software you installed previously to connect to your Direc)
- Select Synchronize item under Jobs and click Enable Synchronization. Note: Synchronization allows the CSM software to update ProxySGs about objects/information that has been changed on your server.
- TIP: If you do not select this option, the CSM sofware will only download the content to your local host, and NOT contact the Director appliance.
- Select "Synchronize all Devices and Groups": The default is to synchronize all devices and groups associated with Director.
- Select Query item under Jobx and click Enable Query to verify that update content was received
- NOTE: The output for the scan will be placed in : C:/Program Files/Expect-5.21/bin/Output folder. This folder only gets created when you do your first scan.
- The scan may take up to 20 mins to complete, depending on the speed of the server you choose, and the amount of data scanned.
- On a successfully scan, the output log will look something like this, except for the enable password errors. The example below is for a HTTP scan, not a CIFS scan.
Points to note:
The latest CSM log keeps track of the latest version of each object on the chosen website. In other words, the CSM does not download, cache, and push actual content out to the SG, it merely keeps track of what content there is to cache, and hands that list off to each SG, so it can download, and cache the objects. The list is displayed in the tracking window when a job first starts to run, and provides a progress report of every 500 objects scanned or crawled. The log is kept in the data directory where you installed CSM, identified by a timestamp. This log is in non-verbose mode and is called CsmGuiJobs.txt . You can change the default to verbose mode by using Tools>Options>Verbose Mode in the main window.
The CSM Configuration file contains, in the all the settings for one job. Each new job has its own configuration file, located in C:/Program Files/Expect-5.21/bin/data. The first CSM configuration file for the job you create is titled csm.cfg. Each new job has its own configuration file; for example, csm001.cfg, csm002.cfg, and so on. Each time the job is run, the csmXXX.cfg file is output in the data directory with a timestamp, so you can see what changes you made in each running of the job.
The CSM Configuration file,called CSM001.cfg, is kept in the same folder, and should not be edited directly. Most of the settings can be changed through the Management Console standard windows; a few can be made only through the Advanced window of the Management Console. (These few settings generally do not need to be changed; the defaults are usually satisfactory.
The recomended platform for the Content Sync Module, and Expect, is Microsoft Windows XP with service pack 3 installed. There are known problems with this software being installed to Windows 7, and 64 bit Windows 2003 servers.
For details on how to create a job to crawl ( read scan) a webserver, such as http://yahoo.com, see000010193.
For details on how to install the CSM, see 000010802
For details on a known problem with CSM and timezone changes, see 000008918
For a list of Proxy SG version compability with Director SGME 18.104.22.168, see 000013458
For details on what problems you may face launching the Director Managment console Java application, see 000016900
For details on helpful Director command Line syntax, see 000014637