Shadow removal from multi-projector displays via three-dimensional modeling and object tracking

When a person stands between a projector and a display surface a shadow occurs. We show a method to perform shadow removal using camera-based object tracking and a three-dimensional model of a room, which includes cameras, projectors, and flat surfaces. For tracking people, although we use cameras, other methods would work. To obtain the model, we adapted and use existing geometric calibration methods. With the tracking and model information, our algorithm finds in the projector image the region that is occluded by a person. Another projector can then automatically fill the region with equal intensity and identical color to that of the occluded projector. We found that calibration and tracking were accurate, and that the system could correctly and efficiently perform shadow removal, providing a more appealing visual experience to users of multi-projector displays.