Dense Image Space (DIS) description
This is a skeleton description of the
construction and display geometry of a spatially dense
array of images (dense image space, or DIS), which retains and
utilizizes all overlapping image data. Superficially a DIS is similar
to
stitching multiple images into a panorama.
With
a panorama:
1) Typically the viewpoint is identical for each
component image. The viewpoints must
identical for perfect "stitching" (coregistration and compositing) of
photographs of 3-D subject
matter.
2) Overlapping image data is only used for
coregistration of adjacent images, and duplicate information is
discarded or blended.
But with a DIS the goal is to:
1) Acquire component images at an array of
different
closely space viewpoints, on a rectilinear or cylindrical grid (in one,
two, or three spatial dimensions).
2) Retain overlapping image data and allow
display of several adjacent perspectives of the same spatial points.
Conceptually, a panorama can be compared to standing
at one spot and pivoting about your eye to look in a variety of
directions; a panning operation in panorama display software, or
visually scanning a large panoramic print, mimics the visual effect of
this egocentric rotation. The analogous comparison for DIS navigation
would be eye translation and
eye pivoting about any subject point.
Figure 1.
a) Single panorama viewpoint covering a wide field of view.
b) Multiple DIS viewpoints covering a wide field of view. A
panning operation corresponds to a viewpoint changes (three on left)
without change in direction of gaze, while a pivot operation
corresponds to viewpoint changes (three on right) with coordinated
changes in direction of gaze such that the same subject point stays
centered in the field of view.
To
achieve visually smooth translation and pivoting with DIS, a small
image
spacing (a high density image space) is required. But even a moderately
dense image space can be used
to mimic a smooth translation and also present discrete adjacent
perspectives in a contiguous fashion. A DIS cannot be projected onto a
single 2-D surface like a panorama. In a sense a DIS is a
multi-perspective panorama; many panoramas from a variety of
perspectives could be constructed from DIS components. Software image
management and digital display, with large random access memory
availability, is an ideal environment for navigating such a
multi-dimensional image space.
Stereo pair display of DIS data is naturally
accomodated if the grid spacing (distance between image
viewpoints) is chosen well. Stereo pairs
can be constructed simply by always choosing horizontally adjacent
pairs of viewpoints while maintaining the same subject points in the
fields of view of the pair.
A simple dense
image space; acquisition, image, and display
geometry
Conside a set of photographs, all taken in the same
direction but from horizontally adjacent veiwpoints (Fig. 2). In general the fields of
view of these images will overlap, so the images will contain several
perspectives of the same object. If this array of images is very large,
we would like a method for quickly and intuitively navigating the image
space, with as much continuity is possible. To achieve this in software
with a digital display, the image set will be conceptually arranged as
a coregistered "stack" (Fig. 2. c)
at the subject distance, and pan
and pivot operations (Fig. 3)
will allow navigation of all possible
views.
Figure 2. A set of
photographic images acquired from a linear grid of viewpoint locations
forms one
possible dense image space.
a) With a fractional overlap f
= .8 at the subject background distance D, any central point will be covered
by the field of
view of 1/(1-f) = 5 images
(five colored rectangles, with corresponding crosses indicating
respective viewpoint grid locations).
b) Cross section of the angular fields of view of each image
intersecting the subject background (gray). The lower mauve ball,
although occluded in the central image, is contained within all five
images, but the upper blue ball at a distance < D is not fully contained in the far
left and right fields of view.
c) Cross section of all images,
projected at the subject background distance D. The order of the stacking is
arbitrary. At this projected distance (relative horizontal alignment)
image points at the background distance will be vertically aligned
(horizontally coregistered).
d) All images, spread out to
show the relevant image differences. In the central image the blue ball
occludes the mauve ball. The image to the immediate right of center
(green frame) shows the perspective difference of this viewpoint. The
image to the far right shows an increased perspective difference, and
clips central foreground (blue ball) subject matter at a distance < D.
In the one dimensional DIS example of
Fig. 2, with fractional
overlap f =
.8 at the subject
background distance D, every subject point at distance D will be covered by 1/(1-f) = 5 images. This allows
five
discrete rotational views of every central point. If this example was
extended to two dimensions (f = .8
in both dimensions) every
point would be covered by (1/(1-f))2
= 25 images, allowing twenty-five discrete rotational views of
every central point.
The DIS display software will, by default,
present the image with a viewpoint that is closest to
perpendicular at the center of the viewport (display window with
respect to the images). In addition the software will allow discrete
pivoting of the viewpoint to available adjacent perspectives. The
number of
available adjacent perspectives will depend on the
amount of overlap in the acquisitions. Any perspective mismatch between
adjacent images displayed in the same viewport will be minimized by
dynamically adjusting the relative
display position of the images based on an estimated distance to
subject near the boundary.
For software and digital display purposes, all
images that have
coverage
within or close to the current viewport are made available in main
memory. An estimate of the subject distance is made from coregistration
information (from the disparity of corresponding points within the
viewport, centrally weighted) . This subject distance is used to
co-align the images, as in Fig. 2.c.
Generally, at native resolution,
the viewport width/height will be a fraction (~1/2) of each image
width/height, and the viewport will display the single image with a
viewpoint that most closely corresponds to that viewport's pan position
and desired pivot angle (Fig. 3).
Figure 3. Image
selection and viewport display with pan and pivot operations. The
available and displayed images are indicated with brown/green/red. The
viewport (blue double stripes, square) is shown with respect to the
field of view of each and all images.
a) Vertical and central
perspective.
b) Right pivot with respect to
a). The same subject matter is in the center of the field of view, but
from an adjacent perspective.
c) Right pivot with respect to
b). The viewport cannot be filled by the extreme right images field of
view, so the remainder is filled with the adjacent perspective's
content.
d-f) A central field of
view pan operation, with respect to a-c), that does not change the
perspective of the image, only the portion of the image within the
viewport. This is similar to panning within an image editor.
g-h)
A edge of field of
view pan operation, with respect to d-e), that does change the
perspective of the image.
i) No image is available at this
extreme pan and pivot "edge" of DIS. A pivot from h) will not be
allowed, and a pan from f) will result in the closest perspective
available at that pan location.
A panning operation that shifts the most
closely
corresponding viewpoint (e.g. Fig. 3,
d/e to g/h respectively) will cause a viewpoint
shift (an adjacent
coregistered image to be displayed) during
the pan operation, minimizing the visual disruption caused by the
perspective change of the viewpoint shift. A pivot operation
(e.g. Fig. 3, a/d/g to b/e/h)
causes an immediate viewpoint
shift. At extreme pivot angles, a composite of two or more
images will be
required to fill the viewport (e.g. Fig.
3. c), introducing a
perspective mismatch across the boundary. The amount of this
perspective mismatch will
depend on the depth of the subject along the boundary
and the viewpoint grid spacing and lens angular field of view geometry.
[[ insert Fig. 4. Stereo pair display of DIS ]]
In the one dimensional DIS example of
Fig. 2, with five
discrete rotational views of every central point, four adjacent
stereo pairs are available. If this example was extended to two
dimensions, twenty (4x5) adjacent stereo pairs at this
orientation
would be available, and many more would be available at other
orientations (20 more at a
90 degree rotation).