Abstract

Depth movies provide 2-D representations of a time-varying 3D scene. Multistream depth movies arise in many structure-capturing setups and have been used for image based rendering. They are bulky and carry redundant information. We propose a proxy-based compression scheme for multistream depth movies of a scene involving dynamic human actors. The input to our system is depth movies from different viewpoints and the calibration parameters to relate depths to 3D points. We use an articulated human model as a proxy to represent the common structure of the scene. The proxy is parametrized by various bone angles.

Abstract

Graph cuts has become a powerful and popular opti-mization tool for energies defined over an MRF and havefound applications in image segmentation, stereo vision,image restoration, etc. The maxflow/mincut algorithm tocompute graph-cuts is computationally heavy. The best-reported implementation of graph cuts takes over 100 mil-liseconds even on images of size640×480and cannot beused for real-time applications or when iterated applica-tions are needed. The commodity Graphics Processor Unit(GPU) has emerged as an economical and fast computationco-processor recently. In this paper, we present an imple-mentation of the push-relabel algorithm for graph cuts onthe GPU. We can perform over 60 graph cuts per secondon1024×1024images and over 150 graph cuts per secondon640×480images on an Nvidia 8800 GTX. The time foreach complete graph-cut is about 1 millisecond when only afew weights change from the previous graph, as on dynamicgraphs resulting from videos. The CUDA code with a well-defined interface can be downloaded for anyone’s use

Abstract

Capturing dynamic scenes in 3D using a suitable depth or structure recovery mechanism has become popular today for image based modelling, 3D telelconferencing, etc. The 2 1 2 D geometric structure and the aligned texture – together called a depth image – are captured by such setups. Time varying sequences of depth images are called depth movies. Depth movies of humans performing interesting actions are being captured using specialized setups today in different research labs. The data so captured is often streamed to another location for immersive viewing using standard depth-image rendering techniques. The depth maps are bulky and need innovative algorithms for efficient representation and compression. In this paper, we present a compression scheme that uses parametric proxy models for the underlying action. We use a generic articulated human model as the proxy and the various joint angles as its parameters for each time instant. The proxy model represents common prediction of the scene structure for that time instant. It can be projected to each view and the difference or residue between the depth due to the proxy and the original depth can be used to represent the scene. The residues compress better and can exploit temporal and spatial relationship between multiple depth movies. We show results on several synthetic actions to demonstrate the usefulness of the scheme

Abstract

Augmenting virtual objects between real objectsof a video poses a challenging problem, primarily because itnecessitates accurate knowledge of the scene’s depth. In thispaper, we propose an approach that generates an approximatedepth for every pixel in the vicinity of the virtual object, which weshow is enough to decide the ordering of objects in every image.We also draw a similarity between layered segmentation andaugmentation. We argue that augmentation only needs to knowwhich layers are in front of the object and which are behind it.Our algorithm makes effective use of object boundaries detectedin an image using image segmentation. The assumption thatthese boundaries correspond to depth discontinuities provides auseful simplification of the problem, sufficient to produce realisticresults. We use a combination of segmentation and featuredetection for sparse depth assignment. Results on challengingdata sets show the effectiveness of our approach

Abstract

Capture of dynamic events is an active research area to-day. Capturingthe212D geometric structure and photomet-ric appearance of dynamic scenesnds applications in 3Dtele-conferencing systems,3DTVetc.The captured ìn depth movies îcontain aligned sequences of depth maps and tex-tures and are often streamed to a distant location for immer-siveviewing. The depth maps are heavy and needed acient compression schemes.In this paper, we present a scheme to compress depth movies of human actors using a parametric proxy model for the under lying action.We use a generic articulated human model as the proxy and the various joint angles a sits parameters for reach time in stant to represent a common prediction of the scene structure. The difference or residue between the captured depth and the depth of the proxy represents the scene to exploits patial coherence. Dif-ferences in residues across time are used to exploit temporal coherence. In tra-frame code frames and difference-coded frames provider and om access and high compression.We show results on several synthetic and real actions to demon-strate the compression ratio and resulting quality using a depth-based rendering of the decoded scene

Abstract

Terrains can be rendered efficiently with rectangular 2D grid of heights. Terrain on spheres, on the other hand, can be rendered using Hierarchical Triangular Mesh (HTM) but the representation does not fit directly with 2D grid of heights. We present a unified representation of HTM and clip mapping using Hexagonal Geome-try Clipmaps. This provides one to one correspondence of vertices and heights, low and constant memory usage, less storage space requirements, no pole singularity, fast vertex look-ups, and large range of view distances. Hexagonal clip maps fit equilateral trian-gles of HTM and achieve uniform triangle count on the screen. We keep the clipmaps on the GPU for fast access from shaders and usevertex buffer objects for fast triangle rendering.

Abstract

Terrains are challenging geometric objects for real-timerendering and interactive manipulation. State-of-the-artterrain rendering systems use custom, multiresolution, rep-resentations like geometry clipmaps for fast rendering onthe GPU. In this paper, we present a system that exploitsthe power and flexibility of the modern GPUs to store, ren-der, and manipulate terrains with minimal CPU involve-ment. The central idea is to use a regular-grid represen-tation and fixed size blocks/tiles that change in resolution.The potentially visible portion of the terrain is cached atthe highest necessary resolution and is rendered from theGPU. The CPU sends a light geometry template which isexpanded by the Geometry Shader to the triangles, usingthe heights stored in the GPU Cache. The CPU performsa coarse culling of the tiles with the GPU performing fineculling. The GPU cache is updated continuously as theviewpoint changes. Our system enables the terrain to bemodified procedurally or edited interactively on the GPUwith no CPU involvement. The terrain can also interactwith a large number of external objects in real-time entirelywithin the GPU. We achieve a consistent rendering rate of100 frames per second with terrain modification and inter-actions as well as a triangle rate of upto 350 million persecond on an Nvidia 8800 GTX GPU for large terrains, witha CPU load below 10%.

Abstract

The GPUs pack high computation power and a restricted architecture into easily available hardware today. They are now used as computation co-processors and come with programming models that treat them as standard paral-lel architectures. We explore the problem of real time raycasting of large deformable models (over a million trian-gles) on large displays (a million pixels) on an off-the-shelf GPU in this paper. Ray casting is an inherently parallel and highly compute intensive operation. We build a GPU-efficient three-dimensional data structure for this purpose and a corresponding algorithm that uses it for fast ray cast-ing. We also present fast methods to build the data struc-ture on the SIMD GPUs, including a fast multi-split opera-tion. We achieve real-time ray-casting of a million triangle model onto a million pixels on current Nvidia GPUs using the CUDA model. Results are presented on the data struc-ture building and ray casting on a number of models. The ideas presented here are likely to extend to later models and architectures of the GPU as well as to other multi core ar-chitectures.

Abstract

The primitives of point-based representations are in-dependent but are rendered using surfels, which approxi-mate the immediate neighborhood of each point linearly.A large number of surfels are needed to convey the exactshape. Higher-order approximations of the local neigh-borhood have the potential to represent the shape usingfewer primitives,simultaneously achieving higher renderingspeeds. In this paper, we proposealgebraic splatsas abasic primitive of representation for point based models.An algebraic splat based representation can be computedusing a moving least squares procedure. We specificallystudy low order polynomial splats in this paper. Quadraticand cubic splats provide good quality and high renderingspeed using far fewer primitives on a wide range of mod-els. They can also be rendered fast using ray tracing onmodern GPUs. We also present an algorithm to constructa representation of a model with a user-specified number ofprimitives. Our method to generates a hole-free represen-tation parametrized by a smoothing radius. The hole-freerepresentation reduces the number of primitives needed bya factor 20 to 30 on most models and by a factor of over100 on dense models like David with little or no drop invisual quality. We also present a two-pass GPU algorithmthat ray-traces the algebraic splats and blends them usinga Gaussian weighting scheme for smooth appearance. Weare able to render models like David at upwards of 200 fpson a commodity GPU using algebraic splats.

Abstract

We present a non-photo realistic, real-time painterly ren-dering technique for terrains. The painterly appearanceand the impression of terrains is created by effectively ren-dering several brush strokes. The strokes have fixed loca-tions on the surfaces of the terrain during animation to en-able frame to frame coherency. The strokes are rendered asalpha blended sprites in two-dimensions and are orientedalong the slope of terrain analogous to the way artists painton canvas. By exploiting the regular nature of terrain data,we create pre-decided rendering depth orders for primitivesfor any camera orientation.With this, we avoid the neces-sity of sorting the primitives of sprites required for alphablending. We use DirectX10/SM4.0 based shaders to renderstrokes to improve performance. Being distributed on ter-rain, strokes get cluttered when they are closely located onscreen. We follow a level of detail scheme that maintains auniform stroke density in screen space. Various styles canbe achieved with different stroke variations. Phong shadingthe rendered output in real-time is possible for more variedstyles. We achieve painterly rendering in real-time with acombination of object space positioning and image spacerendering of strokes. We illustrate our method with imagesand performance results.