ReplaceMagic Configuration

Previous Top Next

ReplaceMagic Configuration is done over "Configuration" button:

Configuration is split on different tabs:

Configuration - here you can configure core options of ReplaceMagic (for example, number of parallel documents to be processed)
Processing - consists of options that will be directly used during processing of documents
Document Properties - used to control which document properties will be shown in Scan document results grid and be included in export. Number of fields will be extended constantly
Security - consists of options to be used in case that documents are password protected
Exports - consists of options that have influence on export functionality
Logs - under this tab you can configure what should ReplaceMagic do with log files
SharePoint Configuration - used to set parameters important during direct connection with documents stored on SharePoint
Notifications - used to set configuration for email notifications (currently only Command Mode run)
Misc - placeholder for additional configuration not fitting to any other tab

All options are described below.

Tab Configuration

Import Separator - set what will be splitter between Search&Replace and Password strings. In case that you have | as a separator then in import files strings should be in format SearchString|ReplacementString
Find documents Import Separator - set what will be separator between fields in Find Documents export file
ReplaceMagic Theme - select design for your ReplaceMagic installation between 16 different themes
Parallel document processing - this setting configures number of documents that ReplaceMagic will process in parallel. This number is limited by hardware power (CPU, memory, IO) and on each computer can be different. Default is 5 but you can experiment and change this number based on hardware usage during processing. Keep in mind that more does not mean faster - if you set 100 but your computer can process only 20 there will be a lot of CPU waits. Use Task Manager to check hardware usage. We also added information how many logical processors you have.
Decrease workload during SharePoint throttling? - if this parameter is set ReplaceMagic will start to reduce workload (number of parallel documents for processing) until not reaching 1 document at the same time or until SharePoint does not stop to throttle processing (no new error message 429 - too many requests are shown)
Optimize workload ... after every ... documents - if this parameter is selected ReplaceMagic will try to increase workload (number of documents for parallel processing) until not reaching hardware capacity (for example, 8 on our test computer )

SharePoint workload safety belt - in case that ReplaceMagic should optimize workload to use maximum what current hardware can do with this parameter we are controlling that when processing is done against SharePoint we do not use more than 8 documents in parallel to reduce risk that ReplaceMagic is throttled by SharePoint. In case your hardware has more than 8 threads and you are using "Optimize workload..." we are recommending to using this parameter:

Ignore Web Logon limit - Per default ReplaceMagic will not allow parallelism higher than 4 documents in case of Web Logon authentication mode. By setting this parameter this limit will be ignored.
Suppress Warning - by setting this ReplaceMagic will show limited number of warnings. This option should be set only if you are experienced ReplaceMagic user.
Enforce memory cleanup => by setting this parameter ReplaceMagic will do memory garbage collection every 100 processed documents reducing used memory.
Enable results filtering => by setting this after processing result grids will have in first row filter fields where you can define what will be visible. This does not have impact on export as it is always complete no matter on set filter.
File size in => used to set in which units file size will be shown in Scan document tab results.
Processing time in => used to set time unit of measure for total processing time shown in scanning or replacement dashboard
Progress bar alignment => possibility to set alignment of text in progress bar
% of memory used when running ReplaceMagic => used to set % of total memory that can be used during ReplaceMagic processing. This counts memory used by ReplaceMagic but also all other processes running. Parameter is introduced to protect that ReplaceMagic does not load too many documents which will fully use memory as this will have performance impacts. For example, in case that you set this parameter to 90 and total used memory is above that value ReplaceMagic will not open new documents until memory is not below this parameter. Memory reduction can happen if document processing is done and memory is released or "Maximum document processing time" parameter value is reached and ReplaceMagic stops processing affected document(s) as that will close memory consumption.

We do not recommend to reduce this parameter to less than 80%. On computers with 8GB (or less) recommendation is to keep parameter to 100.

Maximum time to process document - this parameter will instruct ReplaceMagic to close processing of every document which is started for more than selected number of seconds. This is triggered only in case that there are no threads to start new document processing (message: "Processing opened documents..." or "Waiting on free memory..."
Wait time to close document processing - this parameter instructs ReplaceMagic to wait X seconds before it shows processing results. All documents that were still in process when timeout is reached will be written to AdditionalInfo tab.
Skip files bigger than ... MBytes - in case that some value <>0 is set ReplaceMagic will skip files bigger than that value.
Show scanning dashboard => after scanning is done ReplaceMagic will show dashboard with different data like overview of scanned document types or similar
Add docs age stats to scanning dashboard => by setting this (btw. Show scanning dashboard has to be enabled) you'll see split of all documents at selected location according to create, last accessed and last modified date
Show replacement dashboard => after replacement process ReplaceMagic will show dashboard with replacement overview

"Re-read configuration after XXX documents" => parameter which is controlling if configuration file should be red during documents processing. Currently, ReplaceMagic works in a way that it will read configuration file when you start ReplaceMagic. With this change you can change configuration during processing and after defined number of documents ReplaceMagic will re-read configuration file and pickup new values. For this to work during processing parameter has to be enabled before you start with processing.
Fast Calendar Step - with this parameter you can set how many months ReplaceMagic will move when pressing << or >> in form used to set date filters. Default is 12 months.

Tab Processing

Update ReadOnly Documents - this setting will instruct to make changes also in ReadOnly documents
Skip Temp files (~$*.*) - sometimes there are Office temporary documents. By setting this ReplaceMagic will skip those documents
List processed folders in Console view - by setting this during processing ReplaceMagic will constantly update Console view with currently processed folder
List processed documents in Console view - by setting this during processing ReplaceMagic will constantly update Console view with processed documents. This has small performance impact but provides immediate response what is processed.
List all skipped documents in Additional Info tab - if this option is not selected ReplaceMagic will list only documents that were really processed in case that they were skipped (for example, document locked by somebody, document size in bigger than selected max. size,...). With this parameter ReplaceMagic will list also documents that were skipped as their document type (extension) is not on list of selected extensions. Basically, with this setting you will see all documents that are on selected location(s).
Do not change document attributes - in case if you want to keep document attributes like "Last accessed date", "Last modified date" set this option. If it is set ReplaceMagic will access and possibly change your documents but will keep original document attributes
Check does Hyperlink exist? - by selecting this option ReplaceMagic will check do all if found hyperlinks (HTTP and HTTPS) exist. This has performance impact!
Check does file link exist? - by selecting this option ReplaceMagic will check do all found files (file links in hyperlinks and OLE links) exist or not
Open document with path longer than 260 characters - due to slight performance impact this option is disabled. If needed let us know and we will enable it. Windows has limitation that it cannot open documents if path is longer than 260 characters. To go around this limitation we introduced this feature. If this option is disabled all documents where path is longer than 260 characters will be reported as not accessible.
Include file owner name in results - by setting this ReplaceMagic will include file owner in scan document results but also in other result tabs (except Search&Replace tab)
Check URL Type - MSXML or Check URL Type - WinHttpRequest - is used to check is found URL link correct or not. In first case (MSXML) library MSXML2.XMLHTTP is used and in second case library WinHttpRequest. Which option to use depends on your network configuration
Number of HTTP connections - per default only two connections are used to check is hyperlink ok or not. Over this parameter you can influence how many connections should be used.
UserAgent - as some Web servers require UserAgent you can enter it here. This setting works only for WinHTTPRequest. Example: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36"
URLCheck Sleep time parameter is used to set number of milliseconds that ReplaceMagic will wait before remote server response during URLCheck is returned. This is useful in case that there is slow network connection to remote server. Default value is 0.
HTTP Request timeout - by setting value for this parameter you can define how long ReplaceMagic will wait on remote server to return info is hyperlink ok or not. If timeout happens ReplaceMagic will report that hyperlink is broken.
Proxy settings - in case that you use proxy and method to check broken links is WinHTTPRequest you can provide proxy details which will be used to send check requests. To enable proxy configuration select "Use Proxy" checkbox.
Skip TextToDisplay in PDF documents (memory decrease) - in case that TextToDisplay is enabled in case of bigger PDF documents memory can grow a lot. To disable this we included this option so ReplaceMagic will skip TextToDisplay.
Update Read Folders status (not for productive usage) - by setting this parameter when you click on Scan Documents you'll see documents that are added for processing. Without this parameter you'll see only "Reading folders". You can also specify after how many document would you like to see progress bar refresh. DO NOT set this to very small number to avoid additional refresh overhead.
This setting is not recommended for productive usage as has performance impact.
Continue replacements after first change? - per default, when first replacement is done (hyperlinks, OLE links, Link Source, Pivot tables) ReplaceMagic will stop processing of already changed link. By setting this parameter ReplaceMagic will try to make change for each search&replacement string which is provided.
Do not change formulas in Excel documents - by setting this parameter if ReplaceMagic is changing your Excel documents we will instruct Excel not to refresh formulas (content of cells) after you open in Excel changed document.
Show all Excel Link Sources - In Excel documents under Link Sources there might be link sources that are visible over Data => Connection and Link Sources which are hidden. Previously, ReplaceMagic was showing all Link Sources which is now changed that ReplaceMagic shows only visible Link Sources. In case that you want to see all Link Sources go to Configuration => Processing and select parameter "Show all Excel Link Sources". Only shown Link Sources will be used later for replacements.
Check only XML stored Link Sources - Excel link sources are stored in XML structure until /xl/externalLinks/_rels location but when we check if they are broken or not we will read this location and also location which is used in Excel if you go to Data => Edit links. To avoid false-positive check results we include feature to disable check of links which are shown in Excel. Per default we are checking both location and if one is working ReplaceMagic will report that link is ok.
Stop replacements if backup does not work - if you select that ReplaceMagic should make backup of your documents (all or only changed documents) if that does not work (for example, not enough space to create backup) changes done in document will not be saved. Of course, all skipped documents will appear in Additional Info tab. Default value is selected.
Ultra fast document scan (no extra data) - with this setting you are instructing ReplaceMagic only to add found documents in Find Documents result table. This will be very fast but you will not get any document content information (hyperlinks, OLE Links, Link Sources or Pivot Tables).
URL Decode links for link checker - with this parameter ReplaceMagic will do URLDecode of link which might help that they are better interpreted by servers where they are. This is features is in case that you have links like "sites/0128/Document%20Starter/S%C3%A9quelles/file%20name%207.docx" which are after URLDecode converted to "sites/0128/Document Starter/Séquelles/file name 7.docx"
Do not change file links if replacement is longer than 235 characters - in case if replacement string is longer than 235 characters change will not be done. Reason is that Office cannot read longer path so it might report that document is corrupted. To avoid that set this parameter.
Show Windows Shortcut properties in Hyperlinks tab? - enable ReplaceMagic to show Windows Shortcut document properties in Hyperlinks tab. Following properties will be shown (only if they have some value): Arguments, Description, Hotkey, IconLocation, TargetPath and WorkingDirectory.
Convert Visio (VSD/VSS/VST) docs in VSDX/VSSX/VSTX format? => as ReplaceMagic cannot save changes in VSD, VSS and VST Visio files we provided functionality to convert older Visio file formats to newer formats like VSDX, VSSX and VSTX. By setting this option ReplaceMagic will scan older Visio documents, make replacement (if needed) and save changed documents in newer formats. In case of network drives you will see old and new Visio file plus, eventually, backup of old file if you set that ReplaceMagic should make backup. In case of files stored on SharePoint only old and new file will be present.

Tab Document Properties

Include file owner name in results/export? - control will file owner be included in Scan documents tab and export
Include template name in scan documents results/export (Word)? - control will template name be included in Scan documents tab and export
Include template name in scan documents results/export (Word/PowerPoint)? - control will template path be included in Scan documents tab and export

Tab Security

Password to open document - some documents are password protected; to open them you need a password; use this field to assign password which ReplaceMagic will use when it finds password-protected document
Password to change document - some documents have "change mode" password; use this field to provide password that ReplaceMagic will use to unprotect documents
Password Import - in case that you have list of passwords, for different documents, you can upload it here and ReplaceMagic will try each of provided password to unprotect processed documents

Search for file with passwords. They have to be in text (ASCII) file in following format:

OpenDocumentPassword|UnprotectDocumentPassword
For example:

My first password to open|my first password to unprotect
My second password to open|my second password to unprotect

Import Separator from Configuration tab will be used as a separator of passwords
After you select password(s) file click on Import Passwords button and you right hand side you will see all imported passwords
After clicking on OK button you will get confirmation message and will be back to configuration form

Try multiple passwords - to enable ReplaceMagic to use imported list of password this option has to be set.
After activating this feature ReplaceMagic will always try all imported passwords when it tries to open password protected document. If none of passwords is correct document will be reported as unavailable.
In case that fields "Password to open document" and "Password to change document" are empty ReplaceMagic will skip password protected documents saving some time as they will not be tried to be opened.

Tab Exports

File Export Type - here you can define export file file format

Tab Logs

Create log file? - by setting this option ReplaceMagic will create log file during scan or search&replace process. Log will have Console tab entries.
Developer log info (enable only on ReplaceMagic team request) - additional log details will be stored in log files. This might be needed for deep dive investigation of any issues.
SharePoint developer log info (enable only on ReplaceMagic team request) - additional log details will be stored in log files. This might be needed for deep dive investigation of any issues when processing documents stored on SharePoint.
SharePoint performance metrics (enable only on ReplaceMagic team request) - with this parameter ReplaceMagic will show you how long different phases during processing of documents stored on SharePoint took. This will add additional entries in ReplaceMagic log file and is excellent to understand how time is spent but also to figure if you are throttled as Microsoft has to protect SharePoint environment.

Here you can find entries for one document when Microsoft throttled processing (during normal processing we see between 4 and 10 better performances for same document):

Binary Load Document: document.docx: 2143,2041 ms (comment: this is where we download documents from SharePoint)
Preparation for Meta Data processing: document.docx: 1270,1174 ms (comment: access to the SharePoint)
Replacement process: document.docx: 0,9958 ms(comment: processing done on your computer)
CheckIn/Out or Publish/Unpublish update: document.docx: 2007,0992 ms (comment: access to the SharePoint)
Preparation for save: document.docx: 89,7238 ms(comment: processing on your computer)
Upload to SharePoint: document.docx: 3177,0676 ms (comment: access to the SharePoint)
Set document Metadata back: document.docx: 16490,9978 ms (comment: access to the SharePoint)

Set Logs Location - text field where you can type where logs location should be or you can click on "Select folder for logs" which will open Folder Selector form where you can use tree view to select folder for logs
Open logs location - will open folder where logs are
Create dump file registry entries - by creating those registry entries in case of ReplaceMagic crash Windows should create dump files required by ReplaceMagic for root-cause analyses. Entries will be created under:

Windows will set that full dump is created and that there are no more than 10 dumps in CrashDump folder. Oldest dump will be always overwritten.
Remove dump file registry entries - this will remove registry entries

Currently active log file has name like ReplaceMagicLog.log. All older logs will have in addition time stamp in the name

Tab SharePoint Configuration

Keep "Last modified date" and "Modified by" values => in case that you want to keep SharePoint properties you'll have to select this option. This will instruct ReplaceMagic to keep last modified date and name of person making last changes.
If this is not done after making replacement in your documents you'll see that last modification date is changed to date and time when ReplaceMagic made change and that ModifiedBy is changed to the user that you used to login to SharePoint from ReplaceMagic.
If this option is selected and ReplaceMagic is making changes we will keep all original last modified date and ModifiedBy data.
Important remarks:

ReplaceMagic can keep original SharePoint data only if user used to login to SharePoint from ReplaceMagic belongs to Site Administrator collection. This is limitation of SharePoint.
In case that you want to import previously scanned documents ModifiedBy value cannot be set back to original value and this field will be updated with user data of person running ReplaceMagic. In other words if you want to keep ModifiedBy value do not use import files functionality.

Should ReplaceMagic check-out/check-in documents? - by selecting this option ReplaceMagic will check-out all documents that we are accessing during replacement process and check them in after replacement is done. This will ensure that documents are block for changes of other users during ReplaceMagic processing.

Enforce check-in? (all documents checked-out by other users will be checked-in for processing) - by selecting this option if ReplaceMagic is accessing some document that is check-out by some user ReplaceMagic will check-in that document and make processing. Of course, during processing we will check-out/check-in document.

Remark: when document is forcefully checked-in ReplaceMagic will enter check-in comment "ReplaceMagic forced check-in".

In case that this option is not selected and we try to access some of checked-out documents that document will appear in list of not processed documents with comment "Document checked-out by SOME_USERNAME and cannot be changed!". This will give you enough information to understand what is the user blocking document for processing in case that you want to contact that person.
Query Row Check (ReplaceMagic default value: 4900, no more than 4900 files can be read in one pass; use this fields to set your configured limit)
Per default SharePoint is not allowing that more than 5000 list items are shown/selected. To go around this limitation you'll have to set this parameter specifying when should we merge selection results. This means, if default of 5000 documents is configured, ReplaceMagic will return list of documents in sets of 5000 and always append one set to previously read list of documents. This will allow that we can access also libraries with more than 5000 documents without any limits.

Btw. as this parameter is configurable please check with your SharePoint administrators to what value you should set this parameter!
Keep content approval status? - in case that your document library has selected setting "Require content approval for submitted items?" ReplaceMagic can keep document approval status. This means if documents where with status like Approved, Rejected... we will keep them. As SharePoint API does not allow that approval is changed in the name of some other users if this option is selected we cannot keep "Last Modified Date" and "Editor" data. Those values will be always overwritten with change time stamp and username of person who is running ReplaceMagic.
Keep published status? - in case that SharePoint document library has setting:

if this option is not selected all changed documents will go into minor version and will be unpublished. In case if you set this option documents with minor version will stay in minor version and documents in major version (which are published and visible) will stay in major version. Keep in mind that due to SharePoint API we cannot keep old major version number (it will increase) and we cannot keep last modified date and editor data.
Set versioning? - if you enable versioning for your SharePoint document libraries we can, based on SharePoint settings, make appropriate changes as set in ReplaceMagic. If you select that you want to keep version number this can be done as long as "keep published status" is not selected as in that case if you have major version version number will be increased due to SharePoint API.
All SharePoint Libraries? - by setting this on ReplaceMagic will try to access all SharePoint locations. This means that we can change also system files (like .ASPX) so be extremely careful as you might damage SharePoint platform. Remark: in some cases SharePoint might not allow changes of some files. ReplaceMagic cannot overcome that limitation set by SharePoint.
Set ReplaceMagic comment - per default after check-in/check-out, document publish or unpublish or document approval ReplaceMagic was putting entry in comment field. Now you can disable that ReplaceMagic stores comments or you can specify own text.
HTML Decode/Encode SharePoint pages - use this setting if you have legacy SharePoint pages (like Enteprise Wiki) as content there is HTML encoded thus ReplaceMagic cannot find links (link which normally looks like <a href="some_URL">SomeText</a> will be stored like <a href="some_URK">SomeText</a>&. Use it with caution as wrong decision to use it might corrupt pages.
Unicode Decode/Encode SharePoint pages - use this setting if you have expect to have some Unicode characters in your SharePoint/Web pages Use it with caution as wrong decision to use it might corrupt pages.
SharePoint Legacy Auth Mode - SharePoint supports legacy and modern authentication mode. Depending on your configuration select correct value. Btw. this setting can be set also directly in form where you can enter SharePoint logon details.
Max doc ID value (default:30.000.000) - when ReplaceMagic is searching for max. docID this parameter will set up to which number search will go. Max. ID that SharePoint supports is around 30.000.000 thus this value. If in your document library have ID which are smaller to accelerate search you might decrease value of this parameter but to be on a save side you can leave it as it is.
Process List Item, Quick Launch and Top Nav Bar? - ReplaceMagic can also make replacement in list item fields of value URL, quick launch and top navigation bar. If this is needed enable this parameter. In case that list item has attachment or you are scanning document libraries (they are also list items) processing of URLs in documents will have priority. To make changes of list item hyperlink fields do not run them with processing of documents in document libraries or list item attachments at the same time.
Native SharePoint check of broken link - by selecting this option when check if links are broken or not for documents stored on SharePoint.Online (on-prem versions are not supported) ReplaceMagic will use native SharePoint functionality. Problem is that this will be slow and as for each of links we will need to create SharePoint request this might overload SharePoint leading that Microsoft will throttle complete processing. We do not recommend usage of this option but still want to give decision to you.
This will work only for links that are inside of same SharePoint site where you connected. If link belongs to some other side check will not work as we do not have authentication for this different site.

In case that link is point to other site check cannot work as for this other site different SharePoint context is required and we do not have it. Also this might open need that unlimited number of SharePoint contexts are required which is not feasible.

Long story short - we have this feature but do not recommend usage of it.
Process all skipped documents - By setting this parameter ReplaceMagic will prepare all skipped documents for reprocessing. Without this parameter only documents skipped with following code will be reprocessed:

Important:

All settings changed by ReplaceMagic during processing will be set back to original values after processing is done.
If documents where checked-out and we check them in they will stay checked-in. We cannot check them out with previous user.
To be able to keep last modification date, author and editor user used by ReplaceMagic has to belong to Site Administrator Collection. Otherwise we cannot make changes.
If you set "Keep content approval" and or "Require versioning?" => prerequisite is that option "Should ReplaceMagic check-out/check-in documents?" is also checked. This is done for security reason to avoid conflicts when working with documents.

Tab Notifications

Parameters:

Mail Server - enter your mail server name
SMTP Port (default: 25) - in case that you use some other port enter it here
POP Port (default: 110) - in case that you use some other port enter it here
Sender Email - email from with email will be send (usually has to be email of mail account (username/password)
Receiver Email - email to which notifications will be send. If this field is empty email will be sent to sender email address
Domain - if you are using Exchange (Exchange Web Service) you might need to enter your user domain
Username - enter username (usually your email address) for your email account
Password - enter password for your email account
Encrypt mail username & password - as configuration is saved as plain text ReplaceMagic can encrypt username and password before saving them to the configuration file
Button "Show Password"- by clicking here ReplaceMagic will unhide/hide text in password filed.
Button "Send test mail" - by clicking there we will try to send email to entered email address. After try you will see message telling you if all worked or not.
Mail Server type:

SMTP - standard mail server settings
Exchange (EWS) - use in case that you have Exchange server

Due to different mail server configurations in case of issues let us know. Please provide details like:

Do you use Proxy?
Do you use special ports?
Do you use special SSL configuration?

Tab Misc

Prefix if starts with - if link, which should be checked, is starting with this parameter ReplaceMagic will prefix in front of it value of "Relative Link Prefix"
Relative Link Prefix - in case that ReplaceMagic finds relative link starting with what is entered under parameter "Relative Link Prefix" ReplaceMagic will use value from this field to append in front of matching link

Links checker - log writer - by selecting this option ReplaceMagic will write in log file every link where append happened. You'll find entry like:
Links checker log writer -> Original link: Address, link with append (to be checked): appendLink

Slash Conversion

Slash to backslash - in case of conversion if link has slashes they will be converted to backslashes and link with this format will be checked
Backslash to slash - in case of conversion if link has backslashes they will be converted to slashes and link with this format will be checked

Remark: changes here will not make changes of your links as this conversion is used only for "check broken link" functionality. In case that links should be replaced this has to be done, like until now, through replacement process!