1. Record Managers in the corporate world now want their film scanned even faster and more accurately to minimize confusion and interruptions in document accessibility.
2. The Library of Congress has set up a program called the National Digital Newspaper Project to have all the newspapers from 1836 to 1922 be available on line to preserve our heritage.
3. Special Libraries and educational institutions at all levels are on the move to preserve important/historical documents.
4. OCR is now in most cases expected to be part of the total conversion to make it easier for their employees, clients, patrons or customers to access data. 

The question is how are we supposed to meet these expectations and do it economically? You may already have a number of older film scanners in your facility or have access to them but because they have older/slower technology, the labor costs alone make it very expensive for scanning and then Quality Control. In some cases, the scanning project becomes cost prohibitive. 

Imagine if new scanners could offer you the following: 

1. Faster cameras that make it possible to scan up to 1,000 images per minute accurately while producing even higher quality images.
2. Replace old technology lamp light systems with the latest in high tech LED Light Illuminating Systems that gives a more consistent light while running cooler and requiring less power.
3. Advanced lenses that allows for better quality is small type such as small print in contracts, notary public seals and small print in newspapers.
4. 12K cameras that give you 4,096 levels of gray scale versus 256 levels of gray scale on 8K cameras. Imagine how important this would be for capturing written data on genealogical documents, written signatures and dates on important business documents, photographs and the potential of higher OCR success! 

Well I’m here to tell you that all of this is now available in the newest film scanners. 

1. Film scanning can be done quicker and more accurately without additional labor.
2. QC time is reduced as these tools scan higher quality images.
3. OCR time can be reduced because of higher accuracy. 

The advances in film scanners make it possible to revisit those projects that once were cost prohibitive. Not only is it now economically feasible to scan this film but also offer even more spectacular results in image quality. The future is here!

The most important metric for a scanner is the total throughput for fully processed rolls into satisfactory images per the user’s specification and how much operator time this requires. This information cannot be easily found from the manufacturers specifications as every application is different, and at times the information available can be misleading.  Furthermore, when all stated vendor specifications are reviewed, are you really comparing apples to apples? 

Beware speeds quoted on a manufacturer product datasheet.  Many times, these are quotes based on very best case scenarios (the perfect film with simplest scanning parameters), or these quotes can be numbers that are not realistic for any scenario.  For example, a scanner rated at 325-350 pages/minute might actually only produce about 250 pages/minute using standard film (16mm, 200 dpi, 24x reduction ratio,  8.5” wide image with a 2.5” gap between frames).  The speed quoted on the product specifications may have been declared using a 0 gap between frames, a different reduction ratio or low resolution – all of which may not be realistic for film scanning in a real world scenario.  There are some vendors who think when you specify a 200 dpi image, that they can supply an image scanned at 100 dpi, by 300 dpi and then scale the image to 200 by 200 dpi.  The image quality is abominable, but since they get to scan at 100 dpi in the scanning direction, they get twice the speed and you get half the image quality.  Some vendors call this “double speed”, when they really mean ½ image quality.

Additionally, when quoting speeds for processing entire rolls of film, many times it is not specified for the reader the length of the roll and the type of resolution being employed to arrive at those numbers.

The actual speed of any film scanner is going to always depend on the type of film being scanned and the user’s image requirements.  More importantly, speed should really be measured in terms of the overall scan process, not just how fast the scanner can capture a roll of film.  Scan time, detection editing, image conversion and QC are all relevant factors.

Once the film has been scanned, the majority of the processing is still left to be done.  This is where the differences in productivity can show up between technology providers, and the differences can be huge.

Stand alone scanners where the same computer that is performing the scanning is also doing the processing of the images will typically have about ½ the throughput of the scanner’s “rated” speed. Scheduling the processing to run overnight can help bring the scanner back to its rated speed, but many problems show up overnight that often stop the processing or render the results useless.

The limiting factors here are the hard disk speed, memory, and processing power. For evaluation, doing a system design of the workflow and then running an entire shift on the scanner is prudent,  so that bottlenecks in processing or moving the image data can be determined. Slowdowns may not occur until the hard drive gets fairly full. Data from a hard drive cannot be moved while the scanner is scanning. Scanning and processing cannot be done on the same machine simultaneously at full rated speed without the addition of a high performance storage system.  With the addition of a high performance storage system, processing time can be cut in half in most situations.

With a standalone scanner (no high performance storage system), non-productive operator time waiting for the scanner is approximately 50% – and this is if no rescans need to be performed.  With a networked scanner, combined with true Ribbon Scanning, high performance storage and a second workstation, the software and operator are able to perform detection, processing and QC on a second workstation while the scanner is operating, maximizing operator productivity and scanner throughput.

Welcome to the 21^st century where we want everything right away. Google has built a huge business based on getting information to people quickly and completely. We want a higher speed internet to download documents, music, videos and movies. We want to be able to keep in touch with our friends with Facebook, text them the latest through our cell phone and use Twitter to tweet real time information.

Why then is this Record Manager and many managers out there still trying to support their information backlog with multiple types of stored documents? In the film scanning world, the argument has been that we should only scan 10% to 15% of our backlog because the rest of the information is not accessed that often and besides, the cost of scanning and storing the documents is too high.

The concerns about cost are not valid anymore. Back in 2000, the cost of a GB of data cost $8.37. In 2010, the cost of that same GB of data now costs $.02. The cost of today’s film scanners continues to come down and yet their speed and capabilities have doubled up to 590 images per minute. New “ribbon scan” software captures images off of film quicker and more accurately, and eliminates the need for costly rescans. Improvements in cameras, lenses and Light Illumination Systems account for higher quality images that require less QC time,  saving in both time and money.

A concern that all Record Managers should have regarding their microfilm backfile is the stability of these filmed images. Prior to 1985, the film available for microfilm projects was acetate film. This film was excellent in producing high quality filmed images of paper documents that allowed companies to then destroy the original document to save storage space. We have since learned that it was important to store the film in a controlled environment that monitored temperature and relative humidity. For those companies that didn’t do this, they have since discovered something called the “vinegar syndrome” which indicates that the film is deteriorating. In some cases, the film is in too poor shape to make a duplicate but present day film scanners can come to the rescue because they put minimal tension on the film.

Someone said a few years ago that information is power. That needs to be updated to say he who can retrieve the most complete information quickly, and accurately, has the power. The cost argument for not scanning filmed backlogs just doesn’t hold true today. To not have that information available quickly can make the difference between success and failure.

Kicking off the national Family History Conference at the Salt Palace Convention Center was Wednesday morning’s virtual video tour of the Granite Mountain Records Vault, the seldom-seen site of records preservation and storage for The Church of Jesus Christ of Latter-day Saints.

Granite Mountain is home to some 35 billion images of genealogical information, contained mostly on 2.4 million rolls of microfilm, said Jay L. Verkler, managing director of the church’s Family History Department and presenter of Wednesday’s “tour.”

Some 50-plus employees work at the secluded vault site, where records are stored, copied and digitized.

Granite Mountain doubles as a deep-storage facility protecting materials key to church operations, leadership and history.

If the casual observer sees LDS temples as being semi-secret, then the average Mormon may have similar leanings about the facility located not far up Little Cottonwood Canyon.

But church officials say Granite Mountain’s protective status is not to be secretive as much as secure.

It’s not a unique set-up. Large commercial or government perpetual storage sites are found in states like Kansas, Pennsylvania, Alabama, and Virginia. A private enterprise using similar tunnels is a Little Cottonwood Canyon neighbor.

“Archiving is a trick,” Verkler said. “They [archivists] don’t want patrons touching their records.”

Not just touching, but merely bringing in any contamination, such as “blue jean dust,” the cotton fibers created by brushing pant legs when one walks.

“In our business, we don’t want foreign materials — we don’t want the dust off their shoes, we don’t want the fibers off their clothing,” said Brent Thompson, director of the Church History Department’s preservation services.

“It’s a preservation facility,” he added. “If we are marching visitors through frequently, it would be hard to maintain temperatures and keep it under control.”

In the late 1930s, the LDS Church’s Genealogy Society of Utah began microfilming genealogical records. After amassing more than 100,000 rolls through the next decade and a half, a permanent storage site was needed.

Church officials considered preservation and storage operations as part of an $8 million, 15-story archival building on the corner of Main and North Temple streets, where the new Church History Library now sits.

“It was a great blessing that didn’t happen,” said Thompson, explaining mid-’50s storage technology was limited and construction lacked seismic codes.

Other potential sites included City Creek and Red Butte canyons, until an engineer/architect living in a cottage in Little Cottonwood Canyon remembered its sheer, vertical granite cliffs. He suggested tunnels into the rock might provide maximum protection and temperature stability.
Testing in the late ’50s prompted the church to go full bore; blasting and excavating began in May 1960.

Crews drilled the arched tunnels 10 feet at a time, using ammonium sulfate and diesel fuel to blast every other night. On alternate days, the crews hauled out excess rock.
After tunneling nearly 700 feet without encountering cracks or fissures, the excavation met two challenges — water and a different type of rock. Crews couldn’t go farther.

“Finding that water was very valuable — it was a very big blessing,” said Thompson, noting water is used for processing and washing off microfilm as well as in restrooms and heating and cooling systems.

The first microfilm rolls were transferred in April 1963, even though construction wasn’t completed until the following December, when the facility was open for public tours.

Visitors described the three long, narrow corridors and four cross tunnels, while media reports noted the $2 million price tag. Some 675 feet of granite rock guarded the tunnels from above, and reinforced entrance doors weighing 9 to 14 tons each were said to be able to withstand a nuclear blast.

Steel and concrete lined the tunnels, with banks of metal storage cabinets reaching 10 feet high. The arched interiors were painted in pastel colors, the Deseret News reported, “to alleviate the monotony and eliminate the cavernous atmosphere.”
The site was fully operational in 1965, named in May 1966 and dedicated on June 22, 1966.

Forty-plus years later, the vault boasts the world’s largest collection of family history information.

Microfilm masters — negatives used for duplication and digitization — occupy 60 percent of Granite Mountain’s space. Duplicate rolls are sent to the Family History Library, family history centers and patrons to the tune of 4 million images a week. Digital images are indexed and used in online research.

Digitization started in 2002 as a tedious process. Scanners of that era had trouble reading under- and overexposed images as light densities varied even on the same microfilm roll, meaning operators had to closely watch for problems, stop, back up, readjust and repeat.

Conversion to digital images was projected then to take more than a century, Verkler said.

The church has worked with international scanner manufacturers to develop improved scan rates and procedures. “We think were going to be producing 10 times the images we were five years ago,” Verkler said.

Thompson cites two advantages provided by the granite mountainside vault — a protected environment from intruders, fire, earthquake and disaster as well as a stable storage environment. Improvements in 2001 led to constants of 55-degree temperature and 35 percent humidity.

Besides genealogical preservation and storage, Granite Mountain Records vault serves as the deep archives for a myriad of church materials — including scriptures in every language published, large leather-bound temple ordinance books that were hand-kept through the 1960s, materials and minutes from presiding priesthood quorums, financial records, backup tapes and audio-visual masters from “Legacy” to “Johnny Lingo.”

“Once the records are there, they generally don’t leave there,” said Thompson.

Nor likely seen or touched either.

The answer is a Ribbon Storage Device or simply RSD. This is a separate piece of equipment that is nonetheless an integral part of a total scan system. It is configured to stream down up to 80 MB of data per second and to eliminate any bottle necks during the scan process for peak performance. Depending on the amount of film scanned per hour, these RSD’s typically come in 2 TB, 4 TB, 8 TB and 16TB sizes to accommodate any storage requirements. By including an RSD in the final scan system configuration, there is an additional benefit of being able to QC all the images and write out the final image to its desired format without stopping the scanner! The RSD allows a film scanning group to speed up the overall scan process and still provide both accurate and high quality images at the end of the day.

There are many servers out there that could act as an RSD, but because of a lack of transfer rate speed to the RSD, it would hurt the production rate. What you will discover is that film and fiche scanner manufacturers have tested many servers and offer the one that gives the best results for your specific application. Thus, when a scan system is ordered with an RSD, it is fully factory tested and covered under warranty to minimize any installation/operation problems. All upgrades in software are considered not only for the scanner but also for the RSD to minimize any conflicts or degradation of performance. In short, the RSD is another tool/feature being offered by some film and fiche scanning manufacturers to help increase the profitability of film and fiche scanning.

 To be able to scan microfiche has always been a bigger challenge than film. First, there are 5 different types of microfiche which are Step & Repeat Fiche, Film Jackets, COM, AB Dick and Microx, each of which have their own characteristics. To further complicate the issue, the concept of Film Jackets makes it possible that each row of that jacket might have been filmed in different years on different cameras, which results in mixed quality of each image. The good news is now through improvements in technology, what used to be the difficult or impossible is now possible at a reasonable and affordable price.

Imagine a microfiche scanner that can scan almost as fast as a high production rollfilm scanner. Imagine it can scan for almost an entire shift without any operator intervention and still give you the desired high quality images at the end of the day – without any being lost. Imagine the cost of scanning images on microfiche is now low enough to justify all conversion projects that you previously could not even consider.  There is now technology in the marketplace that is making all of these things possible as a result of dramatic leaps in upgraded/new software, cameras, light systems, optics and hardcore mechanical engineering

The current buzz words when capturing images off of film or fiche are “ribbon scanning”. This innovative way of scanning film or fiche ensures that no images are missed during scanning to eliminate costly rescans. Combine ribbon scanning software with new auto feed mechanisms on Microfiche Scanners that now can hold up to 1,000 fiche at a time and not just 70 fiche – it makes possible the ability to scan for a full shift with little to no operator attention. Cameras have improved so you can scan faster and pick up more detail than ever before. New light illumination systems have been developed to replace the traditional lamp generating less heat and in some cases offering 100,000 hours Mean Time Between Failure – essentially the life of the scanner.

As you can imagine, in addition to hardware improvements, another difference between a good scanner and great scanner is software. New and more powerful tools to improve image quality utilizing gamma correction, filters, new cropping algorithms and flat field correction has only helped to bring out the fine and faded characters. Flat Field Correction is probably something you have never heard of before, but it can make a huge difference in your image quality. Imagine you are standing in a dark room and you are shining a flashlight on a painting on the wall. You can see the center of the painting very well but as you go away from the center, it gets darker and darker. What “flat field correction” does is expand the light of that flashlight so it shows the entire painting at the same light intensity. This ensures that no matter how big the image is, there is even lighting over the entire image to maximize image quality.

Combine the above software functionality with a new fully automated fiche scanner, and what used to seem insurmountable or too expensive is now possible when scanning microfiche.

There is still a large archive of 16mm and 35mm microfilm in the world left to be brought into the digital age.  With the desire to make collections and information available to the public online, the increased affordability of storage and advancements in scanner technology for processing of degraded and difficult images, many organizations are now able to look for the best solution to scan their microfilm and microfiche – and there are a number of manufacturers creating products to do this.

With multiple solutions available on the market today, it is important to understand all features and factors needed for your application, and how best to evaluate offerings from any manufacturer.  Not only is hardware important – software capability, real throughput, image quality and support should all be key considerations, as they will contribute to your overall satisfaction and translate directly to your return on investment.

The purpose of this article is to provide the reader a better idea of what to watch out for and the right questions to ask when evaluating a microfilm scanner – ultimately to assist you in selecting the best solution for your specific application.

Microfilm scanners continue to evolve.  Initially, the traditional method of scanning had been to detect one image at a time and capture just that image.  The latest evolution for scanning now presents some methods of scanning that allow an entire roll of film to be scanned as one image  (called Ribbon Scanning), or for portions or strips of a roll to be scanned as a single image.  The method of capture (Ribbon Scanning/entire roll as one image, strip/section/portion of images on the roll) will depend upon the manufacturer. Today, most production microfilm scanner manufacturers have added some level of this type of capability.

Scanning of more than one image at a time requires more resources, computer processing and storage than the older single image capture method, however, it provides significant advantages over the traditional method of capture as well.

Technology that scans an entire roll as a single image, “Ribbon Scanning,” ensures that there are no missed images. This had been a significant problem for traditional single image capture scanning where the image detection had failed, and the image was not properly captured. This improper capture was then discovered in the QC process and the film had to be remounted, the image located, and then manually captured.   Ribbon Scanning ensures that no images are lost, so no time-consuming rescans are needed, and operator time and intervention is minimized.

With Ribbon Scanning, all film can be scanned – even film of very poor quality. This has been a major source of frustration for both manufactures and users. Film with very narrow gaps between images, film with multiple borders, and film that had poor contrast could not be detected properly using single image capture scanning – so ultimately could not be scanned and digitized. Ribbon scanning scans the raw film and then manual or systematic image detection is used to detect, define and enhance the film images.

Given this shift in how images are processed and technological developments in scanning hardware, what should you take into consideration when evaluating a scanner for purchase?

to read the complete article, please download from the IDMI website (13MB PDF)

http://www.idmionline.com/nextscan/Paul%20Futscher%20-%20Evaluation%20of%20Microfilm%20Scanners%20-%20A4.pdf

It wasn’t that long ago that customers were only interested in converting up to 15% of their microfilm to digital which represented their most recent documents. This microfilm was filmed on the more versatile cameras that could adjust for different colored documents and the filmers had improved lighting systems to eliminate shadows on the edges. Thus, the film was in good enough shape that only minimal image enhancement was required to achieve a good quality image. Also keep in mind that storage was quite a bit more expensive back then, which many times prohibited backfile conversion. As an example, back in 1988, the cost of a MB of storage was $50, while today that same MB costs less than 2 cents!

 Over time, the cost of storage has gone down significantly and now there is a demand to scan more of a customer’s backlog of film. Some film that goes as far back as the early 1950’s is not a pretty sight. Quality is from fair to poor. Rolls of film can deviate a lot in density and contrast from the beginning of a roll to the end. Images can be touching or overlapping. The images can be so faint or so dark that it is difficult to detect where each document begins and ends. There is film that is in poor physical shape due to being stored in areas where there is no climate control. All these conditions have challenged high end scanner manufacturers to come up with a scanner that can handle these microfilms and do it cost effectively.

 Scanning today’s film with older scanners just doesn’t work well and in some cases there is film that simply can’t be scanned using older scanning software. Three years ago, a new, innovative way to capture images off of film was created, called “Ribbon Scanning”. This new software approach does not look at each individual image on a piece of film, but rather scans the entire roll or microfiche edge to edge in 10 bit grayscale thus assuring that no images are missed. Along with this, more powerful QC tools were introduced to maximize image quality and allow for more accurate OCR results.

 None of this would have been possible without vast improvements in motherboards, video cards, image capture cards, cameras and network interface cards. Innovative changes in the light systems offer new LED technology that allows 100,000 hours MTBF versus the incandescent light system in older scanners that has only 2,000 hours MTBF. The biggest benefit from these new light illumination systems is that they bring out the fine detail of written data, notary public seals and small type that can be found in such images as newspapers, legal documents, engineering drawings and maps.

 If you are considering the purchase of used equipment, here are some things to consider.  If some or all of these factors come into play, the REAL cost of your used scanner could actually be more than a new scanner at the end of the day!

•  Some software licenses are non-transferrable.  It is possible that the software for the scanner can only be licensed to the original purchaser of the unit.  It is possible that your software could no longer work should you purchase used equipment, and you may incur additional cost for a new software license.
•  Many times software is not upgradeable without a license.  New licenses for software upgrades can vary in price, dependent upon the need for necessary hardware upgrades within a unit and current version of software installed.  The older the unit, the more cost could be incurred for a software upgrade license.
•  Beware false advertising for used units!  Many dealers may promise fully upgraded/refurbished units, along with ongoing support – only to disappear after the sale.  Many times only the manufacturer is actually authorized to issue software licenses, provide upgrades and/or ongoing product support contracts, or provide spare parts
•  Many times, in order to be eligible for software upgrades and service and support programs, purchased used equipment must be inspected and re-certified by the manufacturer, which can incur additional cost as well

 So is used equipment really a good deal? Because many of the used scanners don’t have the tools to deliver good image quality on film being scanned today, or will require serious work to get the scanner in the condition needed to fulfill your requirements, what you save on the initial cost of the scanner will in all likelihood cost you more in man hours and upgrades to achieve the desired image quality, if you are even able to do so.

 Additionally, older scanners may cause limitations on jobs that can actually be processed.  In the case of a service bureau, this means the possibility of not being able to compete for certain jobs.  With the additional resources that may be required to employ an older scanner, the cost per image that has to be charged for a job will go up, again making the user less competitive for available work.

The ultimate question is do you want a new scanner that works easily and effectively for you or do you want a used scanner that will at the end of the day have you working harder with no guarantee of success?

FamilySearch volunteers expect to have transcribed more than 325 million names by the end of 2009, just three years after the organization began its online indexing program.

The milestone was a number once thought impossible to reach in such a short period of time. In 2006, a few thousand volunteers indexed only 11 million names. But thanks to continuing advances in technology and a growing number of volunteers — more than 100,000 across five continents — an estimated half-million individual names are indexed each day…….

 Film scanner manufacturers have been demonstrating dramatic increased speeds in film scanning at trade shows, but is this realistic? In order to achieve these speeds at these shows, a roll of film has been specially prepared to highlight the speed of the scanner. The images are of high quality and there is minimal spacing between images. Optimal lens reductions at 24x and resolutions as low as 150 DPI are used to highlight this speed feature without any consideration of image quality.

In the real world, the true speed of a scanner is determined by the requirements of a scanning project. Lens reduction and resolution will in fact dramatically affect scan speed. The lower the lens reduction and resolution, the faster the speed and, conversely, the higher the lens reduction and resolution, the slower the speed. It’s great to go as fast as you can, but if you have to rescan over 20% of the film, then maybe it would be more prudent to increase resolution, which would slow down the scanner, but get the desired image quality the first time around. When you are evaluating a new scanner, you also need to consider the quality of the light systems, cameras, lenses, film transport systems and the depth and completeness of the software. This is an investment in your company and you need to be sure that it will not only do the current job you have in house, but that it also has all the features to handle new, challenging jobs in the future,

 So like when you are buying a car, take some time to look under the hood and make sure it’s the scanner for you. I encourage you to take the time for a test drive of the scanner and use your film so that you can measure the true performance of a film scanner. At the end of the day, he who can offer the best image quality on a timely basis wins!