Box Detection and Box Removal

From Grooper Wiki

Box Detection and Box Removal are two closely related IP Commands in Grooper. Both are designed to identify checkboxes in document images, such as those found on forms, surveys, or ballots. Both commands generate layout data describing the position, size, and check state (checked or not checked) of detected boxes. This layout data is required for downstream OMR (optical mark recognition) extraction. However, while Box Detection only locates checkboxes and determines their check state, Box Removal will also remove checkboxes from the image, cleaning up visual noise that may interfere with OCR.

⚠

Box Detection and Box Removal only detect rectangular boxes. They cannot detect radio buttons, OMR, bubbles or circular OMR "boxes".

However, Labeled OMR performs an "on-the-fly" OMR pass that can detect circular OMR boxes.

What is Box Detection?

Box Detection is an IP Command that detects checkboxes and determines their check state (checked or unchecked). The detected checkbox information is stored as part of the page's layout data.

Box Detection is an IP Command in Grooper. It can be added to a step in an IP Profile or IP Group. The Box Detection command is designed to identify and analyze checkboxes within document images. This is a foundational step for optical mark recognition (OMR) workflows, providing the layout data required by OMR extractors such as Labeled OMR, Ordered OMR, and Zonal OMR.

When the Box Detection command is executed, it:

  1. Preprocesses the image (including binarization and filtering) to enhance box visibility.
  2. Analyzes connected components (called "blobs") to identify rectangular regions matching the expected size and aspect ratio for checkboxes.
  3. Determines the check state (checked or unchecked) of each detected box based on the fill ratio of its interior.
  4. Saves layout data describing the position, size, check state, and welded status of each box.

Box Detection does not alter the original image. Its primary output is the set of detected box objects, which are used by downstream OMR extractors to read checkbox values.

Use cases for Box Detection

Grooper can use the layout data generated by Box Detection to:

  • Enable OMR extraction for checkboxes on forms, ballots, or surveys.
  • Provide layout data for Labeled OMR, Ordered OMR, Zonal OMR, and OMR Reader extractors.
  • Support validation and review of detected checkboxes using diagnostic overlays and logs.

General configuration steps

To use Box Detection:

  1. Right-click an IP Profile or IP Group to add a Box Detection IP Step.
  2. Select "Add Command" then "Feature Detection" then "Box Detection".
  3. Adjust detection settings as needed:
    • "Size Range" – Minimum and maximum size for detected boxes.
    • "Minimum Aspect Ratio" and "Maximum Aspect Ratio" – Controls how square or rectangular a box can be.
    • "Maximum Border Thickness" – Limits how thick the border of a box can be.
  4. Adjust "Image Preprocessing" settings as needed:
    • These settings optimize checkbox visibility, improving Box Detection's ability to detect checkboxes. Preprocessing adjustments only occur prior to detecting checkboxes and do not alter the final image.
    • "Binarization Settings" – Controls how color and grayscale images are converted to black and white. Box Detection must occur on black and white images.
    • "Filtering" – Controls how much noise filtering is applied before detection. Filtering can help smooth out edges and remove small imperfections that may interfere with detection.
  5. While the default settings work well for most scenarios, you should test the IP Step/IP Group/IP Profile using the "Tester" tab. Use the Diagnostics panel's images and files to visualize results.
    • Ensure lines are detected as you expect. Review the "Lines.jpg" diagnostic. Detected vertical lines are highlighted in blue. Detected horizontal lines are highlighted in green. A "box" must have all horizontal and vertical lines connected with no gaps.
    • Ensure checkboxes and check states are detected as you expect. Review the "Boxes.jpg" diagnostic. Detected unchecked boxes are highlighted in red. Detected checked boxes are highlighted in green.
    • Fine-tune the Box Detection settings as needed.
  6. The Box Detection step will execute as part of the IP Profile's execution flow.

The detected box information is stored as layout data, making it available for downstream OMR extraction and validation.

What is Box Removal?

Box Removal is an IP Command that detects checkboxes, determines their check state (checked or unchecked) and digitally removes them from an image. The detected checkbox information is stored as part of the page's layout data.

Box Removal is an IP Command in Grooper. It can be added to a step in an IP Profile or IP Group. Box Removal has two purposes:

  • Like Box Detection, it identifies checkboxes in document images, generating layout data for downstream OMR extraction.
  • Box Removal goes one step further by digitally erasing detected boxes from the image. This can eliminate unwanted checkbox content before OCR occurs, improving recognition accuracy and downstream processing.

When the Box Removal command is executed, it:

  1. Preprocesses the image (including binarization and filtering) to enhance box visibility.
  2. Detects checkboxes using the same configurable detection logic as Box Detection.
  3. Classifies boxes as free-standing or welded (attached to lines or other objects). Box Removal can be configured to remove welded boxes or not remove them.
  4. Generates a dropout mask to cover each selected box (all boxes or only free-standing, depending on configuration).
  5. Removes the masked regions from the image using the "Dropout Method" property (e.g., white fill or background fill).
  6. Saves layout data describing the position, size, check state, and welded status of each box.

Box Removal is most useful for cleaning up forms with checkboxes prior to OCR, ensuring that only relevant content remains on the image.

Use cases for Box Removal

Box Removal's use cases include all those of Box Detection, plus:

  • Cleaning up document images before OCR to prevent checkboxes from interfering with text recognition.
  • Redacting checkboxes if not needed/wanted on the final exported image.

General configuration steps

To use Box Removal:

  1. Right-click an IP Profile or IP Group to add a Box Removal IP Step.
  2. Select "Add Command" then "Feature Removal" then "Box Removal".
  3. Adjust detection settings as needed:
    • "Size Range" – Minimum and maximum size for detected boxes.
    • "Minimum Aspect Ratio" and "Maximum Aspect Ratio" – Controls how square or rectangular a box can be.
    • "Maximum Border Thickness" – Limits how thick the border of a box can be.
  4. Adjust "Image Preprocessing" settings as needed:
    • These settings optimize checkbox visibility, improving Box Detection's ability to detect checkboxes. Preprocessing adjustments only occur prior to detecting checkboxes and do not alter the final image.
    • "Binarization Settings" – Controls how color and grayscale images are converted to black and white. Box Detection must occur on black and white images.
    • "Filtering" – Controls how much noise filtering is applied before detection. Filtering can help smooth out edges and remove small imperfections that may interfere with detection.
  5. Adjust removal settings as needed:
    • "Dropout Method" – Controls how masked regions are filled. There are two Dropout Methods to choose from:
      • Fill (Default) - Replaces checkboxes with a solid color. This method is sufficient for most cases when removing boxes for OCR (when Box Removal is applied by an OCR Profile).
        • By default, the Fill Color is set to "none". When set to "none" Grooper will detect a color from the image's background and use that.
      • Inpaint - Digitally restores masked regions by estimating and filling in missing or damaged areas using advanced algorithms. When removing boxes that overlap text, this method can help preserve overlapping text.
    • "Remove Welded Boxes" – Controls whether boxes attached to lines or other objects are removed.
  6. While the default settings work well for most scenarios, you should test the IP Step/IP Group/IP Profile using the "Tester" tab. Use the Diagnostics panel's images and files to visualize results.
    • Ensure lines are detected as you expect. Review the "Lines.jpg" diagnostic. Detected vertical lines are highlighted in blue. Detected horizontal lines are highlighted in green. A "box" must have all horizontal and vertical lines connected with no gaps.
    • Ensure checkboxes and check states are detected as you expect. Review the "Boxes.jpg" diagnostic. Detected unchecked boxes are highlighted in red. Detected checked boxes are highlighted in green.
    • Ensure boxes are removed as you expect. Review the "Dropout Mask.jpg" and "Output Image.jpg" diagnostics. The "Dropout Mask.jpg" image shows the dropout mask in red. The "Output Image.jpg" image shows the final image, as altered according to the "Dropout Method" settings.
    • Fine-tune the Box Removal settings as needed.
  7. The Box Removal step will execute as part of the IP Profile's execution flow.

The detected box information is stored as layout data, making it available for downstream OMR extraction and validation.

Similarities and differences between Box Detection and Box Removal

Similarities

The big takeaway: Both Box Detection and Box Removal detect checkboxes and generate layout data.

  • Both commands detect checkboxes (OMR marks) in document images.
  • Both generate layout data describing the position, size, and check state of detected boxes, which is required for downstream OMR extraction.
  • Both use the same detection technology and support the same range of pixel formats.
  • Both can be configured with the same detection properties, including size range, aspect ratio, adn border thickness.
  • Both provide diagnostic output to assist with configuration and troubleshooting.
  • Both are included as steps in an IP Profile, allowing them to be combined with other image processing operations.

Differences

The big takeaway: Box Detection does not alter the image. Box Removal does alter the image.

  • Both Box Detection and Box Removal detect checkboxes and store their information as layout data.
  • Only Box Removal removes detected boxes from images, masking them out before further processing.
    • This is most useful for image cleanup prior to OCR.
    • Boxes will be removed from an image permanently when Box Removal is applied by the Image Processing activity.
    • Boxes will be removed in-memory prior to OCR when Box Removal is applied by the Recognize activity.
  • Only Box Removal has the Feature Removal properties "Dropout Method" and "Remove Welded Boxes".

When to use each command

  • Use Box Detection when you only need to extract checkbox information for OMR extraction, validation, and review without altering the image.
  • Use Box Removal when you want to eliminate checkboxes or to remove box-like noise prior to OCR or data extraction. Box Removal will extract checkbox information as well.

About Box Detection settings

Box Detection and Box Removal have several configurable properties. In most cases, the default properties work well. However, each property allows you to fine-tune detection for your specific documents or form layout.

Below is a detailed explanation of each property, including its purpose, usage, and configuration guidance.

Shared Detection Settings properties

Both Box Detection and Box Removal share these properties.

Size Range
Specifies the minimum and maximum size (on either axis) for blobs to be considered as checkboxes. Use unit-aware values (e.g., 6pt, 18pt, 10px, 0.25in). Adjust to match the expected size of checkboxes in your forms.
Minimum Aspect Ratio
Controls how "tall and skinny" a box can be and still be detected. The aspect ratio is width divided by height. A value of 1.0 means only perfectly square boxes are allowed. Lower values allow for taller, narrower boxes.
Maximum Aspect Ratio
Controls how "short and wide" a box can be and still be detected. The aspect ratio is width divided by height. A value of 1.0 means only perfectly square boxes are allowed. Higher values allow for wider, flatter boxes.
Maximum Border Thickness
Limits how thick the border of a detected box can be, relative to its overall size. Specify a value between 0.0 (no border) and 1.0 (border as thick as the box itself). Helps exclude filled or irregular shapes.
Binarization Settings
Controls how the input image is converted to black and white before box detection. Choose from several thresholding methods (e.g., auto, adaptive, dynamic) and adjust parameters as needed. Use diagnostic images to verify that box borders are clearly defined.
Filtering
Determines whether a median cross filter is applied to the image before box detection. Filtering can help smooth out edges and remove small imperfections. Options include None, Low (3x3), and High (5x5).

Box Removal properties

Only Box Removal has these properties.

Dropout Method
Determines how masked regions are filled when boxes are removed. Options include filling with a solid color (e.g., white or background) or using custom fill logic. The selected method affects the visual result and may impact downstream OCR or data extraction.
Remove Welded Boxes
Controls whether boxes attached to lines or other objects are removed. When set to false, only free-standing boxes are removed. When set to true, all detected boxes are removed, including those welded to lines or other features.

Summary Table

Property Purpose Typical Usage
Size Range Minimum and maximum size for a box Match expected checkbox size
Minimum Aspect Ratio How tall and skinny a box can be Allow tall/narrow boxes
Maximum Aspect Ratio How short and wide a box can be Allow wide/flat boxes
Maximum Border Thickness Thickest allowed border Exclude filled/irregular shapes
Binarization Settings Convert to black/white Optimize detection
Filtering Noise filtering Improve detection on noisy images
Dropout Method How boxes are removed Fill with white/background
Remove Welded Boxes Remove boxes attached to lines Clean up all or only free-standing boxes

Configuration tips

  • Start with default values and review diagnostic images to assess detection and removal quality.
  • Adjust properties incrementally, testing on multiple sample documents.
  • Use diagnostic outputs ("Binarized", "Dropout Mask", "Boxes", etc.) to visualize the effect of each property.
  • Fine-tune for your specific document set, balancing thorough box detection with preservation of important content.

When are IP Profiles executed?

An IP Profile is executed whenever Grooper needs to process an image using a defined sequence or hierarchy of image processing operations. Execution typically occurs in the following scenarios:

  • By the Image Processing activity: The Image Processing activity will apply the IP Profile and permanently alter the image.
  • By an OCR Profile: OCR Profiles configured with an IP Profile will run the IP Profile on an image prior to handing it to the OCR engine. The image will not be permanently altered.
  • By the Recognize activity's "Alternate IP" configuration: IP Profiles executed by this configuration will only execute feature detection commands (such as Box Detection) to collect layout data.
  • During a Review step: Users can manually execute an IP Profile from the Thumbnail Viewer (if configured to allow the user to do so).

Execution follows the order and logic defined in the IP Profile, including any conditional flow control or branching. Each step or group within the profile is applied in sequence, transforming the input image and producing results for each subsequent step.

Retrieved from "https://wiki.grooper.com/index.php?title=Box_Detection_and_Box_Removal&oldid=30165"