Side Loads

If you have econtent records already loaded in your ILS, you should be able to export the existing files and side load them into Aspen. However, your eContent vendor should be able to provide you with a new up-to-date file containing all records in their collection. Reach out to the vendor's support for more information. Sometimes, searching "vendor + MARC records" on their website or from a search engine can also help get you in the right place.

The records must be in .mrc, .marc, or .mrk format.

Zip file upload is not working at this time. If you have a .zip file containing many MARC files, please upload your .zip file, then submit a support ticket. We will be able to manually unzip the file for you on the server.

Note: Records must have an 001 field or other field with unique record identifiers.

To turn on the side loads module, go to Aspen Administration > System Administration > Modules > Side Loads > Click Enabled and Save

Go to Aspen Administration > System Administration > Permissions. 

The permission that corresponds to side loads is in the Cataloging & Econtent drop down and is: Administer Side Loads - Controls if the user can administer side loads.

Part 1: Create New Side Load Settings

First, navigate to the Side Loads section of Aspen Administration. Click Settings, then click Add New.

1. Give the side load a name. This will be the display name for the collection in your catalog. Avoid using special characters in the name such as :, &, (, etc.

Example: Kanopy

2. In Record URL Component, type in the name of your side load with no spaces.

Example: kanopy

This tells Aspen how to create the URL for these records.

3. Update the MARC Path field. Leave the text in this field as-is with the exception of {sideload_name} -- delete that portion of the text and replace it with your actual sideload name with no spaces, special characters and remove the brackets.

Example: /data/aspen-discovery/test.production/kanopy/marc

Then, copy the side load name you entered in the MARC Path field and enter this into the Individual MARC Path field. The side load name in these fields must be EXACTLY the same.

Example: /data/aspen-discovery/test.production/kanopy/marc_recs

This is creating the paths for these files to be stored on the server.

4. Save your settings.

Here is an example of a side load setting that has been filled out for Kanopy:

Optional Settings: the following settings are not necessary to change in most cases, but may be of interest depending on your needs/preferences.

Access Button Label: The label for the button users will click to access this resource. By default, this will be Access Online. If you want this to say something else, you can change it here. 

Show Status: Will show availability if checked; this is checked on by default.

Filenames to Include: Leave as-is unless you plan to upload an .mrk file rather than an .mrc or .marc file. If this is the case, copy and paste the following text into this field:    .*\.(ma?rc|mrk)

Format Information: To have the side loaded records display as a specific format, change "Load Format From" to "Specified Value," then enter in the specified format, select the format category, and apply any format boost if desired.

Example:

Part 2: Scope Your Side Load

Once the setting is saved, Aspen will create a corresponding Scope setting. Scope settings tell Aspen which libraries should have access to these records. From Aspen Administration, navigate to Side Loads > Scopes, then click on the corresponding scope to edit. 

1. Toward the bottom of the settings, click Add New to set all the applicable libraries and locations that should have access to this side load. To speed up this process, you can click Apply to All Libraries and/or Apply to All Locations and either leave as-is, or delete any libraries/locations that should not have access.

2. Save your changes.

Part 3: Adding Files

Now that the setting has been created and the scope has been set, you can add files. 

1. Navigate to Side Loads > Settings > edit the setting.

2. Click Upload MARC file.

3. Click Select MARC File to choose and upload the file containing your records. The accepted file formats are .zip, .gzip, .mrc, and .marc. The .mrk format will also work, but requires adjustment of the Filenames to Incude field within the side load settings. 

Important: only check the box for "Replace Existing Files?" if you want to delete ALL previously uploaded files and replace with the file you're currently uploading. Otherwise, leave unchecked for new files and additional files.

4. Click Upload File. Repeat this process for any additional files as needed.

5. Once you successfully add the files, they will go through an initial indexing process. You can check the status of this in the Indexing Log within the Side Loads menu to confirm. The records will show up in the "Products Added" column. Then, you'll need to go back into the side load settings, check the box for "Run Full Update" and save.

Checking this box forces Aspen to index the records again. This time, the records will show up in the "Products Updated" column, and the items will now show in the catalog search results.

Add A New Scope

In Aspen, "scope" refers to "where the settings should be applied." In some cases, you may need to create multiple scopes for the same side load settings. Some common scenarios include:

• You have one set of Kanopy records, but each library will need their own unique access URLs applied. Rather than uploading what is essentially the same collection multiple times, you can upload the files once, then create multiple scopes with URL matching and replacement.
• You have a regular catalog and a children's catalog. For the children's catalog, a scope can be used to "include children's materials only."

To add a new scope:

1. Head to Side Loads > Scopes > click Add New.

2. Select the appropriate side load setting. 

3. Enter a name for the new scope.

Examples:

• If you have multiple catalogs and the URLs of the side load will be different, you may want to name your scopes according to the location names -- ie, Main Branch, East Branch, and so on -- one for each seperate catalog. 
• If you have a seperate kid's catalog, you may want to have a scope for "All Records" and another for "Children's Records Only"

4. Adjust/enter in any settings as needed.

5. Click Save Changes and Stay Here.

6. Before leaving, select the Libraries and Locations this scope should apply to. Click Add New to add the libraries/locations indivually. To speed up this process, especially when you have many locations, click Apply to All Libraries and/or Apply to All Locations to automatically apply to all libraties/locations. If you need to edit out locations, you can do so by deleting them individually.

7. Save your changes.

You can also verify that the settings are correct for each catalog by navigating to Aspen Admin > Primary Configuration > Library Systems > click into the library system setting. Scroll toward the bottom to Side Loaded eContent Scopes, where you should see all side loads you have assigned to this library system.

Add A New Scope

As of the 24.04 release, Aspen now has the ability to scope side loads by age/audience.

To view your options, go to Aspen Administration > Side Loads > Scopes > click into your existing scope or Add New

A common use for scoping this content is for a seperate kid's catalog. Then, the library can show off their full collections in their main catalog and only show off their kid's or teens materials in their other catalog.

The options are:

• Include Adult Titles
• Include Teen Titles
• Include Kids Titles

These options follow the bib level rules in the 006/008. See here. 

Run a Full Reindex

To begin loading the records with your new scope settings applied:

1. Return to Side Loads > Settings > click into the applicable setting.

2. Scroll down and check the box for Run Full Update.

3. Click Save.

You can see the indexing progress in Side Loads > Indexing Log.

Note: If records don't load after the indexing is complete, click into Side Loads > Settings > Run Full Update a second time. It can take two full updates for the records to show. 

Depending on the type of URL in the side load, Aspen has multiple different options for changing the URL. 

You might need to change the URL in the MARC records if:

• Multiple libraries use the same set of MARC files, but each library catalog needs to have a different URL to authorize the records
• You have a generic set of MARC records where you need to add your library's URL instead 

In the side load's scopes, you have a number of options to change the URL. 

Switch the URL with URL to Match and URL Replacement

Example:

In this example, I have three scoped catalogs sharing the same set of Kanopy MARC records. The URL in the MARC records is: https://mainlibrary.kanopy.com

First, I added three individual scopes: "All Records - Main," "All Records - East," and "All Records - West"

For Main, I can leave it as-is because the URLs in the MARC records are already set to go to Main's Kanopy records.

For East and West, I will need to tell Aspen to change the URLs in the MARC records to their respective library's URLs. 

You can do this with either URL components or the full URLs to change. 

In the East example, I told Aspen anywhere in the URL that is says 'mainlibrary', change this to 'eastlibrary'

In the West example, I told Aspen anywhere in the MARC records it sees https://mainlibrary.kanopy.com to change it to https://westlibrary.kanopy.com

Either of these will work. 

Replace URL with EZProxy or Other Authentication Link

To replace an entire URL in the MARC records to an EZProxy or other authentication link, you can replace the URL completely by using regex.

In this example, the URL to Match is set to (.*) to change the entire URL in the MARC records to the URL Replacement that is added which is an EZProxy link.

Replace a Part of a Generic URL

If the URL in your MARC records is generic, you can target certain aspects of the URL.

Example: 

In this example, the URL is https://link.gale.com/apps/PTCP?u=[LOCATIONID]&hub=teacher

You would need to tell Aspen to fill in your location's ID to authenticate this URL.

In this example, URL To Match is (\[LOCATIONID\]) and the URL replacement is the library's location ID. 

Add a Beginning to All URLs 

Example: 

You want to add a beginning to all of the URLs, like http://proxy.com/login?url=

In this example, in the scope, in the URL to Match (Regular Expression) field, put in (.*)

Then, in the URL Replacement (Regular Expression) field, put in the beginning URL with $1 at the end, example: http://proxy.com/login?url=$1

When making URL scoping changes, you will have to run 1-2 full reindexes before the changes apply in your catalog

Depending on your library's workflow and how often the vendor updates or adds new records, you may need to occasionally add new records to your side loads -- perhaps monthly or quarterly. 

To add a new file to your side load:

1. Head to Aspen Administration > Side Loads > Settings > Click into the Side Load > click Upload MARC File. 

To see which files have already been loaded, click View MARC files. Before you make any changes, you may want to view and download the current files as a backup.

2. Click Select MARC File to choose the file containing your records.

IMPORTANT: Clicking "Replace Existing Files?" will delete all existing records and replace them with the new file. DO NOT click "Replace Existing Files?" if you are just uploading new titles and the new file does not include the current records.

3. Click Upload File.

4. Return to Side Loads > Settings > edit the side load setting > scroll down and check Run Full Update. 

5. Double check that your new records are loaded in the catalog. You can check on the side load record indexing progress from Side Loads > Indexing Log.

If the indexing has run and your files have not updated, go to Side Loads > Settings and check Run Full Update again. 

Remove Specific Records

To remove specific records from a side load, go to Side Loads > Settings > click into the side load

In Deleted Records, you can add the record IDs from the Record Number Tag, by default the 001. You can add these one on each line and/or separated by commas. 

Once added, Aspen will ignore these record IDs in the side load MARC file. 

This is easy to manage within Aspen but can become a bit cumbersome as you continue to add more and more records to delete. 

Replace Existing Files

Some libraries prefer to work with one MARC file at a time and upload a new file for the entire collection each time changes are made. 

1. To access your current file, you can click on Side Loads > Settings > click into the side load > View MARC files

2. Click on the hyperlinked file name to download the file.

3. In an editing tool, like MARC Edit, you can remove the records you want to delete. 

4. Once you make all your deletions and save the new file, return to the side load settings and click Upload MARC file. 

5. Select your new MARC file.

6. Check the box for "Replace Exisiting Files?"

Important: This will delete your current side load records and replace them with your newly uploaded file.

7. Click Upload File.

8. Once uploaded, go back to Side Loads > Settings > Scroll down and check Run Full Update, then Save.

Double check that your new records are loaded in the catalog.  You can check your the indexing progress of your records in Side Loads > Indexing Log.

If the indexer has run and your files have not updated, go to Side Loads > Settings and check Run Full Update again. 

If you go into the Side Load and click Delete without first clearing out the records and/or the scopes, you will most likely see an error and may need to contact support.

If you are looking to remove a side load completely, we recommend doing this one of two ways:

Remove the Scope First

First, go to Primary Configuration > Library Systems > click into the library system you'd like to remove it from > scroll to the bottom toward Side Load eContent Scopes > find the Side Load Scope and click Delete.

Then, go to Side Loads > Settings > click into the side load you want to remove > check Run Full Update

Wait until the full update runs. You can check Side Loads > Indexing Log to see the progress. 

Once the update runs, you should no longer see those records in your catalog.

Then, it's up to you to decide what to do with that side load setting. Leaving the setting there won't cause any harm if you're not seeing the records in your catalog. If you don't plan on using the side load in the future, you can go ahead and click into Side Loads > Settings > click into the side load you are removing > Delete

Remove the MARC Files(s) First (best for removing the same side load on multiple library catalogs)

First, go to Side Loads > Settings > click into the side load you want to remove > View MARC Files

Here you will see a list of all the MARC File(s) associated with that side load. Click Delete on each file.

Then, go to Side Loads > Settings > click into the side load you want to remove > check Run Full Update

Wait until the full update runs. You can check Side Loads > Indexing Log to see the progress. 

Once the update runs, you should no longer see those records in your catalog.

Then, it's up to you to decide what to do with that side load setting. Leaving the setting there won't cause any harm if you're not seeing the records in your catalog. If you don't plan on using the side load in the future, you can go ahead and click into Side Loads > Settings > click into the side load you are removing > Delete

To see how many users are accessing side loaded records via Aspen, you can check Side Loads > Dashboard.

In general, if a vendor can provide you with MARC records, they can be side loaded into Aspen. 

Here are some commonly side loaded resources:

• BookFlix Books and Movies
• Brainfuse
• Chilton's
• Comics Plus
• Creativebug
• Credo Reference
• Gale Courses, Gale eBooks
• Flipster
• Freading
• Infobase
• Kanopy
• LinkedIn Learning
• LOTE4Kids
• O'Reilly
• Peterson's Test and Prep Center
• PressReader
• Project Gutenberg
• UnglueIT
• Worldbook

However, the side load function has also been used to load in records for other resources, such as museum or park passes.

Aspen can be set to ignore a number of econtent providers, in favor of using the direct API connection and/or sideloading directly into Aspen. 

To set which eContent records are automatically suppressed from your ILS, go to Aspen Administration > ILS Integration > Indexing Profiles > Suppress Records With Urls Matching.

This means that any records containing URLs matching these domains in the 856u will be automatically suppressed in the catalog.

You can add additional URLs here separated with pipes. 

Once side loads are set up successfully, they will appear in the eContent Collection facet along with any other side loads and any econtent coming in via API. 

If you don't see the eContent Collection facet, check your facet set up in Aspen Administration > Catalog/Grouped Works > Group Work Facets. 

You'll want to make sure the "eContent Collection" facet is added and "Show on Results Page" should be checked.

If this facet is still not showing, you may want to check that your side load is scoped correctly. Head to Aspen Administration > Side Loads > Scopes, edit the applicable scope setting, and make sure that all desired libraries and locations are checked.

You can also check within the library systems to verify the scoping has been set up correctly. Aspen Administration > Primary Configuration > Library Systems > click into the library you want to check > scroll down to Side Loaded eContent Scopes. This will show all side loaded content applied to that libray system. 

If your side loaded records aren't showing up in search results, try the following steps.

1. Verify that your record file(s) uploaded successfully.

Edit your side load settings and click View MARC Files. If your file was successfully added, you will see the file name along with the date added and file size.

If you see a file here but there is no valid extension shown in the file name (.mrc, .marc, etc), double check that your file is in a valid file format and re-upload the file. Sometimes manually adding the valid extension to the file name on your device before uploading will fix the issue.

Note: Zip file upload is not working at this time. If you have a .zip file containing many MARC files, please upload your .zip file, then submit a support ticket. We will be able to manually unzip the file for you on the server.

2. Force a Full Update.

If you verified that a file was successfully uploaded and it has the correct file extension, you may just need to force the records to index again. 

Edit the side load setting, scroll down, check the box for "Run Full Update," then save. The next time the side load indexing process runs, Aspen will reindex the records and they should appear in search results within a few minutes (depending on how big the collection is).

3. Check the Indexing Log.

If you have done all of the above and your records are still not showing, go to Side Loads > Indexing Log and check to see if your side load log has any errors preventing the files from successfully indexing.

If you have just forced a full update, sometimes the process may simply still be running. Large collections can sometimes take up to 15 minutes to finish a full index.

When setting up sideloads, you might come across an error like this in your Side Loads > Indexing Log and notice your files aren't loading:

ERROR: Marc Directory /data/aspen-discovery/library.production/sideloadname/marc did not exist

This error is usually caused by creating a new side load and changing the information in the {sideload_name} in the MARC Path or the Individual MARC Path (example: you changed the {sideload_name} to gale but then you decide you want to change it to galeebooks instead). These fields are telling Aspen where to store this information on the server. If Aspen starts this process and then the side load name is changed again, Aspen will usually display that Marc Directory error. 

You can fix this in one of two different ways:

• First, you could delete that side load setting and start over by creating a new side load with a new, never before used {sideload_name}.
• Second, you could put in a support ticket with your Aspen team to adjust the path on the server.