Sketch2Pose: Estimating a 3D Character Pose from a Bitmap Sketch

KIRILL BRODT and MIKHAIL BESSMELTSEV, Université de Montréal, Canada

SIGGRAPH 2022 (Honorable Mention Award)

sketch2pose
Given a single natural bitmap sketch of a character (a), our learning-based approach allows to automatically, with no additional input, recover the 3D pose consistent with the viewer expectation (b). This pose can be then automatically copied a custom rigged and skinned 3D character (c) using standard retargeting tools (d). Input image © Olga Posukh.

Artists frequently capture character poses via raster sketches, then use these drawings as a reference while posing a 3D character in a specialized 3D software --- a time-consuming process, requiring specialized 3D training and mental effort. We tackle this challenge by proposing the first system for automatically inferring a 3D character pose from a single bitmap sketch, producing poses consistent with viewer expectations. Algorithmically interpreting bitmap sketches is challenging, as they contain significantly distorted proportions and foreshortening. We address this by predicting three key elements of a drawing, necessary to disambiguate the drawn poses: 2D bone tangents, self-contacts, and bone foreshortening. These elements are then leveraged in an optimization inferring the 3D character pose consistent with the artist's intent. Our optimization balances cues derived from artistic literature and perception research to compensate for distorted character proportions. We demonstrate a gallery of results on sketches of numerous styles. We validate our method via numerical evaluations, user studies, and comparisons to manually posed characters and previous work.

Key Words: character posing, rigged and skinned characters, sketch-based posing, character sketches




@article{brodt2022sketch2pose,
    author = {Kirill Brodt and Mikhail Bessmeltsev},
    title = {Sketch2Pose: Estimating a 3D Character Pose from a Bitmap Sketch},
    journal = {ACM Transactions on Graphics},
    year = {2022},
    month = {7},
    volume = {41},
    number = {4},
    doi = {10.1145/3528223.3530106},
}
curl -L http://www-labs.iro.umontreal.ca/~bmpix/sketch2pose/bibtex.bib