Side Loads

 

Side Loads

In this section, learn about:

  • Setting up a new side load
  • Adding files to a side load
  • Deleting records from a side load

  

Aspen can load eContent via direct API connection for:

  • Boundless
  • cloudLibrary
  • hoopla
  • Libby
  • Palace Project

For eContent records from other vendors, libraries can side load them directly into Aspen. Click here for more information about eContent via API.

As of the 24.06 release, the side load settings can also be used for physical collections.

  

Getting MARC Records

If you have records already loaded in your ILS, you should be able to export the existing files and side load them into Aspen. However, your eContent vendor should be able to provide you with a new up-to-date file containing all records in their collection. Reach out to the vendor's support for more information. Sometimes, searching "vendor + MARC records" on their website or from a search engine can also help get you in the right place.

The records must be in .mrc, .marc, or .mrk format.

Zip file upload is not working at this time. If you have a .zip file containing many MARC files, please unzip the files first before uploading.

 

Note: Records must have an 001 field or other field with unique record identifiers.

 

2024-06-07 km bws
Add the Side Load Module and Set Up Permissions

To turn on the side loads module, go to Aspen Administration > System Administration > Modules > Side Loads > Click Enabled and Save

Go to Aspen Administration > System Administration > Permissions. 

The permission that corresponds to side loads is in the Cataloging & Econtent drop down and is: Administer Side Loads - Controls if the user can administer side loads.

Add a New Side Load

Part 1: Create New Side Load Settings

 

First, navigate to the Side Loads section of Aspen Administration. Click Settings, then click Add New.

 

 

1. Give the side load a name. This will be the display name for the collection in your catalog. Avoid using special characters in the name such as :, &, (, etc.

Example: Kanopy

 

2. Decide how you want the button next to the record to display. The default text will be Access Online.

Your choices here are:

 - Define text in the Access Button Label field. i.e. Check out on Kanopy

or

- Check the Use Link Text For Button Label. This will pull from the 856. 

Be mindful of the length of text used in these buttons. 

If you are pulling text from the 856, you can use translation mode to update but it is likely better to update the marc records directly if possible.

 

3. In Record URL Component, type in the name of your side load with no spaces.

Example: learningexpress

This tells Aspen how to create the URL for these records.

 

4. Update the MARC Path field. Leave the text in this field as-is with the exception of {sideload_name} -- delete that portion of the text and replace it with your actual sideload name with no spaces or special characters and remove the brackets.

Example: /data/aspen-discovery/library.production/kanopy/marc

This creates the directories where these files will be stored on the server.

 

5. Save your settings

Here is an example of a Kanopy side load all filled out:

 

Optional Settings: the following settings are not necessary to change in most cases, but may be of interest depending on your needs/preferences.

Access Button Label: The label for the button users will click to access this resource. By default, this will be Access Online. If you want this to say something else, you can change it here. 

Show Status: Will show availability if checked; this is checked on by default.

Filenames to Include: Leave as-is unless you plan to upload an .mrk file rather than an .mrc or .marc file. If this is the case, copy and paste the following text into this field:    .*\.(ma?rc|mrk)

Format Information: To have the side loaded records display as a specific format, change "Load Format From" to "Specified Value," then enter in the specified format, select the format category, and apply any format boost if desired.

Example:

 

Side Load Settings (Physical Collections)

As of the 24.06 Aspen release, libraries can use the side load settings for physical collections as well. This is great for special collections (MediaBank, Kits, Bookable collections, etc.).

Follow the same settings but make sure this box is unchecked for physical collections.

Aspen will look at the 856 and only load links where first indicator is 4, 0, or blank and the second indicator is 0, 1, 4, or blank.

 

Part 2: Scope Your Side Load

Once the setting is saved, Aspen will create a corresponding Scope setting. Scope settings tell Aspen which libraries should have access to these records. From Aspen Administration, navigate to Side Loads > Scopes, then click on the corresponding scope to edit. 

1. Toward the bottom of the settings, click Add New to set all the applicable libraries that should have access to this side load. To speed up this process, you can click Apply to All Libraries and/or Apply to All Locations and either leave as-is, or delete any libraries/locations that should not have access.

Tip! In Primary Configuration > Locations, you can edit the Location settings to automatically refer to the sideloaded content for the parent library. If your locations are set up this way, you can simply select the Library in the side load scope settings instead of also having to select the Locations.

 

2. Save your changes.

 

Part 3: Adding Files

Now that the setting has been created and the scope has been set, you can add files. 

1. Navigate to Side Loads > Settings > edit the setting.

2. Click Upload MARC file.

 

 

3. Click Select MARC File to choose and upload the file containing your records. The accepted file formats are .zip, .gzip, .mrc, and .marc. The .mrk format will also work, but requires adjustment of the Filenames to Incude field within the side load settings. 

Important: only check the box for "Replace Existing Files?" if you want to delete ALL previously uploaded files and replace with the file you're currently uploading. Otherwise, leave unchecked for new files and additional files.

4. Click Upload File. Repeat this process for any additional files as needed.

5. Once you successfully add the files, they will go through an initial indexing process. You can check the status of this in the Indexing Log within the Side Loads menu to confirm. The records will show up in the "Products Added" column. Then, you'll need to go back into the side load settings, check the box for "Run Full Update" and save.

 

Checking this box forces Aspen to index the records again. This time, the records will show up in the "Products Updated" column, and the items will now show in the catalog search results.

 

 

Updated 2024-06-24 km bws
Scope Side Loads

Add A New Scope

In Aspen, "scope" refers to "where the settings should be applied." In some cases, you may need to create multiple scopes for the same side load settings. Some common scenarios include:

  • You have one set of Kanopy records, but each library will need their own unique access URLs applied. Rather than uploading what is essentially the same collection multiple times, you can upload the files once and then create multiple scopes with URL matching and replacement.
  • You have a regular catalog and a children's catalog. For the children's catalog, a scope can be used to "include children's materials only."

To add a new scope:

1. Head to Side Loads > Scopes > click Add New.

2. Select the appropriate side load setting. 

3. Enter a name for the new scope.

Examples:

  • If you have multiple catalogs and the URLs of the side load will be different, you may want to name your scopes according to the location names -- ie, Main Branch, East Branch, and so on -- one for each separate catalog. 
  • If you have a separate kid's catalog, you may want to have a scope for "All Records" and another for "Children's Records Only"

4. Adjust/enter in any settings as needed.

5. Click Save Changes and Stay Here.

6. Before leaving, select the Libraries and/or Locations this scope should apply to. Click Add New to add the libraries/locations individually. To speed up this process, especially when you have many libraries, click Apply to All Libraries and/or Apply to All Locations to automatically apply to all libraries/locations. If you need to deselect any locations, you can do so by deleting them individually.

Tip! In Primary Configuration > Locations, you can edit the Location settings to automatically refer to the sideloaded content for the parent library. If your locations are set up this way, you can simply select the Library in the side load scope settings instead of also having to select the Locations.

 

7. Save your changes.

You can also verify that the settings are correct for each catalog by navigating to Aspen Admin > Primary Configuration > Library Systems > click into the library system setting. Scroll toward the bottom to Side Loaded eContent Scopes, where you should see all side loads you have assigned to this library system.

 

 

Add A New Scope

As of the 24.04 release, Aspen now has the ability to scope side loads by age/audience.

To view your options, go to Aspen Administration > Side Loads > Scopes > click into your existing scope or Add New

A common use for scoping this content is for a separate kid's catalog. This way the library can show off their full collections in their main catalog and only show off their juvenile or teen materials in their other catalog.

The options are:

  • Include Adult Titles
  • Include Teen Titles
  • Include Kids Titles

 

These options follow the bib level rules in the 006/008. Click here to read more about these rules

 

Run a Full Reindex

To begin loading the records with your new scope settings applied:

1. Return to Side Loads > Settings > click into the applicable setting.

2. Scroll down and check the box for Run Full Update.

3. Click Save.

You can see the indexing progress in Side Loads > Indexing Log.

 

 

Note: If records don't load after the indexing is complete, click into Side Loads > Settings > Run Full Update a second time. Depending on the timing of when you modify settings and when the indexer runs, it can take two full updates for the records to show changes. 

 

 

Updated 2024-06-10 km bws
Change URLs for the Side Loads

Depending on the type of URL in the side load, Aspen has multiple different options for changing the URL. 

You might need to change the URL in the MARC records if:

  • Multiple libraries use the same set of MARC files, but each library catalog needs to have a different URL to authorize the records
  • You have a generic set of MARC records where you need to add your library's URL instead 

In the side load's scopes, you have a number of options to change the URL. 

 

 

Please note that the following examples are not exhaustive and may not work for every set of records. We're unable to provide documentation for every possible side load resource. If you need assistance with URL modification, please reach out to your support company.

 

Switch the URL with URL to Match and URL Replacement

Example:

In this example, I have three scoped catalogs sharing the same set of Kanopy MARC records. The URL in the MARC records is: https://mainlibrary.kanopy.com

First, I added three individual scopes: "All Records - Main," "All Records - East," and "All Records - West"

For Main, I can leave it as-is because the URLs in the MARC records are already set to go to Main's Kanopy records.

For East and West, I will need to tell Aspen to change the URLs in the MARC records to their respective library's URLs. 

You can do this with either URL components or the full URLs to change. 

In the East example, I told Aspen anywhere in the URL that is says 'mainlibrary', change this to 'eastlibrary'

In the West example, I told Aspen anywhere in the MARC records it sees https://mainlibrary.kanopy.com to change it to https://westlibrary.kanopy.com

Either of these will work. 

 

 


 

Replace URL with EZProxy or Other Authentication Link

To replace an entire URL in the MARC records to an EZProxy or other authentication link, you can replace the URL completely by using regex.

In this example, the URL to Match is set to (.*) to change the entire URL in the MARC records to the URL Replacement that is added which is an EZProxy link.


Replace a Part of a Generic URL

If the URL in your MARC records is generic, you can target certain aspects of the URL.

Example: 

In this example, the URL is https://link.gale.com/apps/PTCP?u=[LOCATIONID]&hub=teacher

You would need to tell Aspen to fill in your location's ID to authenticate this URL.

In this example, URL To Match is (\[LOCATIONID\]) and the URL replacement is the library's location ID. 

 


Add a Beginning to All URLs 

Example: 

You want to add a beginning to all of the URLs, like http://proxy.com/login?url=

In this example, in the scope, in the URL to Match (Regular Expression) field, put in (.*)

Then, in the URL Replacement (Regular Expression) field, put in the beginning URL with $1 at the end, example: http://proxy.com/login?url=$1

When making URL scoping changes, you will have to run 1-2 full reindexes before the changes apply in your catalog

 

Updated 2023-07-14 km bws
Add Additional Records to a Side Load

Depending on your library's workflow and how often the vendor updates or adds new records, you may need to occasionally add new records to your side loads -- perhaps monthly or quarterly. 

To add a new file to your side load:

 

1. Head to Aspen Administration > Side Loads > Settings > Click into the Side Load > click Upload MARC File

To see which files have already been loaded, click View MARC files. Before you make any changes, you may want to view and download the current files as a backup.

2. Click Select MARC File to choose the file containing your records.

 

 

IMPORTANT: Clicking "Replace Existing Files?" will delete all existing records and replace them with the new file. DO NOT click "Replace Existing Files?" if you are just uploading new titles and the new file does not include the current records.

 

3. Click Upload File.

4. Return to Side Loads > Settings > edit the side load setting > scroll down and check Run Full Update

5. Double check that your new records are loaded in the catalog. You can check on the side load record indexing progress from Side Loads > Indexing Log.

If the indexing has run and your files have not updated, go to Side Loads > Settings and check Run Full Update again. 

Modify Side Load Format

By default, Aspen will use the bib-level format determination rules to determine format for side loads. This configuration is best if the records within a side load contain a variety of formats (eBook, eAudio, etc). Click here to read more about how Aspen determines format for side loads.

To instead set a specific format for all records in the side load, here's how:

  1. Go to Side Loads > Settings > Edit the side load setting you want to modify.
  2. Scroll to the Format Information section.
  3. Under Load Format From, switch this from Bib Record to Specified Value. (Note that the "Item Record" option is not currently a valid option)
  4. If your marc records are for physical records, uncheck Convert Format to eContent
  5. In Specified Format, enter the format label you want to appear for all records in the side load. In the below example, a library has side loaded records for their museum passes and wants the format button to reflect what they are.

  1. Select a format category and/or format boost.
  2. Check the Run Full Update box below this section, then save your changes. After the side loaded records reindex, you should see the format labels updated. You can check on the side load reindexing progress under Side Loads > Indexing Log.

 

2024-06-07 km bws
Delete MARC Records from a Side Load

Remove Specific Records

To remove specific records from a side load, go to Side Loads > Settings > click into the side load

In Deleted Records, you can add the record IDs from the Record Number Tag, by default the 001. You can add these one on each line and/or separated by commas. 

Once added, Aspen will ignore these record IDs in the side load MARC file. 

This is easy to manage within Aspen but can become a bit cumbersome as you continue to add more and more records to delete. 

 

Replace Existing Files

Some libraries prefer to work with one MARC file at a time and upload a new file for the entire collection each time changes are made. 

1. To access your current file, you can click on Side Loads > Settings > click into the side load > View MARC files

2. Click on the hyperlinked file name to download the file.

3. In an editing tool, like MARC Edit, you can remove the records you want to delete. 

 

 

4. Once you make all your deletions and save the new file, return to the side load settings and click Upload MARC file

5. Select your new MARC file.

6. Check the box for "Replace Exisiting Files?"

Important: This will delete your current side load records and replace them with your newly uploaded file.

 

 

7. Click Upload File.

8. Once uploaded, go back to Side Loads > Settings > Scroll down and check Run Full Update, then Save.

Double check that your new records are loaded in the catalog.  You can check your the indexing progress of your records in Side Loads > Indexing Log.

If the indexer has run and your files have not updated, go to Side Loads > Settings and check Run Full Update again. 

Delete a Side Load Completely

If you go into the Side Load and click Delete without first clearing out the records and/or the scopes, you will most likely see an error and may need to contact support.

If you are looking to remove a side load completely, we recommend doing this one of two ways:

Remove the Scope First

First, go to Primary Configuration > Library Systems > click into the library system you'd like to remove it from > scroll to the bottom toward Side Load eContent Scopes > find the Side Load Scope and click Delete.

 

Then, go to Side Loads > Settings > click into the side load you want to remove > check Run Full Update

Wait until the full update runs. You can check Side Loads > Indexing Log to see the progress. 

Once the update runs, you should no longer see those records in your catalog.

Then, it's up to you to decide what to do with that side load setting. Leaving the setting there won't cause any harm if you're not seeing the records in your catalog. If you don't plan on using the side load in the future, you can go ahead and click into Side Loads > Settings > click into the side load you are removing > Delete


Remove the MARC Files(s) First (best for removing the same side load on multiple library catalogs)

First, go to Side Loads > Settings > click into the side load you want to remove > View MARC Files

Here you will see a list of all the MARC File(s) associated with that side load. Click Delete on each file.

 

Then, go to Side Loads > Settings > click into the side load you want to remove > check Run Full Update

Wait until the full update runs. You can check Side Loads > Indexing Log to see the progress. 

Once the update runs, you should no longer see those records in your catalog.

Then, it's up to you to decide what to do with that side load setting. Leaving the setting there won't cause any harm if you're not seeing the records in your catalog. If you don't plan on using the side load in the future, you can go ahead and click into Side Loads > Settings > click into the side load you are removing > Delete

 

 

Updated 2024-06-10 km bws
Side Load Dashboard

To see how many users are accessing side loaded records via Aspen, you can check Side Loads > Dashboard.

Frequently Asked Questions

What can I side load?

In general, if a vendor can provide you with MARC records, they can be side loaded into Aspen. 

 

Here are some commonly side loaded resources:

  • BookFlix Books and Movies
  • Brainfuse
  • Chilton's
  • Comics Plus
  • Creativebug
  • Credo Reference
  • Gale Courses, Gale eBooks
  • Flipster
  • Freading
  • Infobase
  • Kanopy
  • LinkedIn Learning
  • LOTE4Kids
  • O'Reilly
  • Peterson's Test and Prep Center
  • PressReader
  • Project Gutenberg
  • UnglueIT
  • Worldbook

 

However, the side load function has also been used to load in records for other resources, such as museum or park passes.

 

Updated 2023-06-12 km bws
What if these records are already loaded in my ILS?

Aspen can be set to ignore a number of econtent providers, in favor of using the direct API connection and/or sideloading directly into Aspen. 

To set which eContent records are automatically suppressed from your ILS, go to Aspen Administration > ILS Integration > Indexing Profiles > Suppress Records With Urls Matching.

This means that any records containing URLs matching these domains in the 856u will be automatically suppressed in the catalog.

 

 

You can add additional URLs here separated with pipes. 

Where can I find side loaded content?

Once econtent side loads are set up successfully, they will appear in the eContent Collection facet along with any other side loads and any econtent coming in via API. 

Physical side loads will show up in the Available Now At facet.

 

 

If you don't see the eContent Collection facet, check your facet set up in Aspen Administration > Catalog/Grouped Works > Group Work Facets

You'll want to make sure the "eContent Collection" facet is added and "Show on Results Page" should be checked.

 

 

If this facet is still not showing, you may want to check that your side load is scoped correctly. Head to Aspen Administration > Side Loads > Scopes, edit the applicable scope setting, and make sure that all desired libraries and locations are checked.

You can also check within the library systems to verify the scoping has been set up correctly. Aspen Administration > Primary Configuration > Library Systems > click into the library you want to check > scroll down to Side Loaded eContent Scopes. This will show all side loaded content applied to that libray system. 

 

I set up my side load, but no records are showing up in search results. Why?

If your side loaded records aren't showing up in search results, try the following steps.

 

1. Verify that your record file(s) uploaded successfully.

Edit your side load settings and click View MARC Files. If your file was successfully added, you will see the file name along with the date added and file size.

 

 

If you see a file here but there is no valid extension shown in the file name (.mrc, .marc, etc), double check that your file is in a valid file format and re-upload the file. Sometimes manually adding the valid extension to the file name on your device before uploading will fix the issue.

 

Note: Zip file upload is not working at this time. If you have a .zip file containing many MARC files, please try unzipping the files first.

 

2. Force a Full Update.

If you verified that a file was successfully uploaded and it has the correct file extension, you may just need to force the records to index again. 

Edit the side load setting, scroll down, check the box for "Run Full Update," then save. The next time the side load indexing process runs, Aspen will reindex the records and they should appear in search results within a few minutes (depending on how big the collection is).

 

 

3. Check the Indexing Log.

If you have done all of the above and your records are still not showing, go to Side Loads > Indexing Log and check to see if your side load log has any errors preventing the files from successfully indexing.

If you have just forced a full update, sometimes the process may simply still be running. Large collections can sometimes take up to 15 minutes to finish a full index.

 

Updated 2024-06-07 km bws
There is an error that the Marc Directory doesn't exist

When setting up sideloads, you might come across an error like this in your Side Loads > Indexing Log and notice your files aren't loading:

ERROR: Marc Directory /data/aspen-discovery/library.production/sideloadname/marc did not exist

This error is usually caused by creating a new side load and changing the information in the {sideload_name} in the MARC Path or the Individual MARC Path (example: you changed the {sideload_name} to gale but then you decide you want to change it to galeebooks instead). These fields are telling Aspen where to store this information on the server. If Aspen starts this process and then the side load name is changed again, Aspen will usually display that Marc Directory error. 

You can fix this in one of two different ways:

  • First, you could delete that side load setting and start over by creating a new side load with a new, never before used {sideload_name}.
  • Second, you could put in a support ticket with your Aspen team to adjust the path on the server. 

Video Tutorial