Abstract

Human body tracking typically requires specialized capture set-ups. Although pose tracking is available in consumer devices like Microsoft Kinect, it is restricted to stick figures visualizing body part detection. In this thesis, we propose a method for full 3D human body shape and motion capture of arbitrary movements from the depth channel of a single Kinect, when the subject wears casual clothes. We do not use the RGB channel or an initialization procedure that requires the subject to move around in front of the camera. This makes our method applicable for arbitrary clothing textures and lighting environments, with minimal subject intervention. Our method consists of 3D surface feature detection and articulated motion tracking, which is regularized by a statistical human body model [40]. We also propose the idea of a Consensus Mesh (CMesh) which is the 3D template of a person created from a single view point. We demonstrate tracking results on challenging poses and argue that using CMesh along with statistical body models can improve tracking accuracies. Quantitative evaluation of our dense body tracking shows that our method has very little drift which is improved by the usage of CMesh We explore the possibility of improving the quality of CMesh using RGB images in a post processing step. For this we propose a pipeline involving Generative Adversarial Networks. We show that CMesh can be improved from RGB images of the original person by learning corresponding relative normal maps ( N R map ). These N R map have the potential to encode the nuances in the CMesh with respect to ground truth object. We explore such method in a synthetic setting for static human like objects. We demonstrate quantitatively that details which are learned from such a pipeline are invariant to lighting and texture changes. In future the generated N R map can be used to improve the quality of CMesh

Abstract

Keywords: Speech signal analysis; analysis of production dynamics; zero time windowing; heavily decaying window; Hilbert envelope; group delay spectrum; dominant resonant frequencies; glottal activity; open quotient; vowel nasalization; nasal segments; oral nasal tract coupling; Speech signal is the output of a dynamic production mechanism. The articulators involved in the production process move continuously at different rates, giving rise to a time–varying vocal tract transfer function. The process of speech production is dictated by the linguistic and para–linguistic information being conveyed by the speaker. The resulting speech signal embodies the characteristics of this time–varying acoustic system, which is easily perceived by humans. Algorithmic processing of the speech signal attempts to study these characteristics, to uncover the behavior of the acoustic response of the vocal tract system during the production of different sounds. Analysis of the dynamics of the production mechanism from the speech signal is a difficult task due to rapid movement of articulators and complex interaction among different elements of production. The elements of the speech production comprise of static and dynamic articulators. Movement of the articulators can be voluntary, driven by the motor functions (such as the movement of lips, tongue, jaw etc.), to produce a desired sequence of sounds. There are some involuntary or semi– voluntary movements of these articulators that take place due to factors such as the air pressure and muscular tension. Glottal opening and closing, and the velar opening and closing are two such examples of semi–voluntary movements, which are caused during the production of voiced and nasal sounds, respectively. Changes in the acoustic response of the vocal tract system during these events are rapid, and tracking the dynamics of such responses is difficult. Assumption of stationarity or quasi–stationarity behavior of the system limits our comprehension of the production viii ix process. A detailed study of the dynamic nature of the acoustic transfer function is therefore necessary to gain deeper insight into the speech production process. Traditional methods to study the speech production mechanism assume stationarity of the vocal tract acoustic system over a short duration of 10–30 ms. The spectral characteristics for these segments are extracted using methods based on short time analysis of speech. The spectral envelope obtained from the magnitude of the short time Fourier transform (STFT) is interpreted as the frequency response of the acoustic system. The fine structure of the magnitude is attributed mostly to the excitation component of the production system. The gross and fine structure of the spectrum is exploited to develop models for the speech production system. The source–system modeling is a paradigm based on this principle. Analysis using bank of bandpass filters is another method to study the spectral characteristics. The output energy of each filter within a short (10–30 ms) segment is attributed to the spectral energy in the band corresponding to the filter. There is an implicit quasi–stationarity assumption in the implementation of these methods. These methods do not bring out the underlying production mechanism well enough to explain dynamics of the articulatory phenomena. The objective of this study is to examine changes in the vocal tract system to bring out the dynamic nature of the system during production of different sounds. The glottal activity in voiced speech involves opening and closing of the vibrating vocal folds, which results in coupling and decoupling of the subglottal and supraglottal tracts, respectively. The production of nasal consonants and nasalized vowels also involves coupling and decoupling of the nasal tract due to velar movement. These changes occur continuously over a short interval of time, and hence are difficult to track using analysis based on quasi–stationarity assumption. A method is proposed to track the dominant resonance characteristics of the vocal tract acoustic system continuously as a function of time. The method is useful to study the dynamics of the production system caused by the movement of vocal folds, velum, and other articulators. In order to study the dynamic characteristics of the vocal tract system during speech production, a speech analysis method, called the zero time windowing (ZTW) is used. The ZTW method gives spectral characteristics of the time varying system at each instant of time, reflecting the instantaneous behavior of the production system. The spectral characteristics are derived using the Hilbert envelope of the numerator of the group delay (HNGD) function. The behavior of the x time varying system response is captured using the dominant resonance frequency (DRF) obtained from the HNGD spectrum. Changes in the production system due to co

Abstract

Sports video data is recorded for nearly every major tournament but remains archived and inaccessi-ble to large scale data mining and analytics. However, Sports videos have a inherent temporal structure,due to the nature of sports themselves. For instance, tennis, comprises of points, games and sets played between the two players/teams. Recent attempts in sports analytics are not fully automatic for finer details or have a human in the loop for high level understanding of the game and, therefore, have limited practical applications to large scale data analysis. Many of these applications depend on specialized camera setups and wearable devices which are costly and unwieldy for players and coaches, specially in a resource constrained environments like India. Utilizing very simple and non-intrusive sensor(s) (like a single camera) along with computer vision models is necessary to build indexing and analytics systems. Such systems can be used to sort through huge swathes of data, help coaches look at interesting video segments quickly, mine player data and even generate automatic reports and insights for a coach to monitor. Firstly, we demonstrate a score based indexing approach for broadcast video data. Given a broadcast sport video, we index all the video segments with their scores to create a navigable and searchable match. Even though our method is extensible to any sport with scores, we evaluate our approach on broadcast tennis videos. Our approach temporally segments the rallies in the video and then recognizes the scores from each of the segments, before refining the scores using the knowledge of the tennis scoring system. We finally build an interface to effortlessly retrieve and view the relevant video segments by also automatically tagging the segmented rallies with human accessible tags such as ‘fault’ and ‘deuce’. The efficiency of our approach is demonstrated on broadcast tennis videos from two major tennis tournaments. Secondly, we propose an end-to-end framework for automatic attributes tagging and analysis of broadcast sport videos. We use commonly available broadcast videos of badminton matches and, un- like previous approaches, we do not rely on special camera setups or additional sensors. We propose a method to analyze a large corpus of broadcast videos by segmenting the points played, tracking and recognizing the players in each point and annotating their respective strokes. We evaluate the performance on 10 Olympic badminton matches with 20 players and achieved 95.44% point segmentation accuracy, 97.38% player detection score (mAP@0.5), 97.98% player identification accuracy, and stroke segmentation edit scores of 80.48%. We further show that the automatically annotated videos alone could enable the gameplay analysis and inference by computing understandable metrics such as player’s reaction time, speed, and footwork around the court, etc. Lastly, we adapt our proposed framework for tennis games to mine spatiotemporal and event data from large set of broadcast videos. Our broadcast videos include all Grand Slam matches played between Roger Federer, Rafael Nadal and Novac Djokovic. Using this data, we demonstrate that we can infer the playing styles and strategies of tennis players. Specifically, we study the evolution of famous rivalries of Federer, Nadal, and Djokovic across time. We compare and validate our inferences with expert opinions of their playing styles.

Abstract

Forecasting energy consumption enables users to plan their resource utilization optimally. For this,it is essential to establish a reliable forecast of consumption profile in real-time, which is very challenging and still emerging. In this study, we focus on forecasting HVAC cooling energy consumption at a sub-hourly forecasting horizon which enables us to analyze and control the demand in real-time. With the use of energy meters and BACnet technology in the target building (KCIS), we have collected the cooling energy consumption data and HVAC systems usage at a higher resolution in the target building. In this methodology, weather data from Weather Underground of Gachibowli, Hyderabad location has been used. These recorded weather parameters are being used as explanatory variables. Along with the weather parameters, many other feature-engineered explanatory variables which are discussed in the thesis have been used to capture the trend and variation of energy consumption. Required pre-processing and data analysis is performed on the raw data to obtain meaningful information for modeling. We have implemented a one-step-ahead static forecasting using regression and deep learning techniques like Lin- ear Regression, Support Vector Machine (SVM) Regression, K-Nearest Neighbour (KNN) Regression, Gradient Boosting Regression Trees (GBRT), Long Short-Term Memory (LSTM). By comparing the results obtained from using the aforementioned forecast learning algorithms for various combinations of input parameters/explanatory variables, we have carried out a critical model evaluation to attain an ex-emplary forecasting model. All the methods were observed to fit comparatively well. However, GBRT was observed to provide better results marginally and consistently. Our results show that we could improve forecast accuracy by almost 45% by including feature-engineered variables. We have achieved a Mean Absolute Percentage Error (MAPE) of 14.7% with GBRT for the best combination of input features. We observe that at higher values of energy consumption, we are able to forecast with higher accuracy. For energy values greater than 10kWh, the GBRT model was observed to produce precise results with an accuracy of 9.6% MAPE. The model evaluation was done for a data-set ranging from the months of April to August in 2018, as the data of the real building is available during this period. We further evaluate the performance of the model throughout the year based on the data generated from the EnergyPlus simulation software. As calibrating the target building is an onerous and massive task, we design a basic model with realistic set-point, occupancy and internal load schedules. We have achieved a MAPE of 6.7% for the simulated data. Based on recent studies in peak demand reduction, simple measures like increasing cooling set-point temperatures in the case of HVAC systems load management, serves as an effective demand response. Majority of the past studies in demand response focus on developing strategies that reduce peak demand. To practically adapt these strategies, there is a necessity that the user is capable of understanding the impact of these strategies and aware of how to respond, especially during peak demand. Research in estimating the potential of DR programs is gaining momentum these days and a lot of research has to be done to develop reliable applications that can be used in real-time. In the next part of the thesis, we have therefore focused on developing a model that predicts the impact of changing the HVAC set-point temperatures on the cooling energy demand. The model was thus evaluated on the basis of the afore-mentioned simulated data. The model performance was evaluated by changing set-point temperatures for a period of time and comparing them with the results obtained from simulation. An analysis was performed to test the model performance at any given time in a day. We have achieved a MAPE of 7%. In the next part of the thesis, we have demonstrated the application of the prediction model for the demand response. This is a simple and user friendly application using which the decision makers or the building energy engineers can know how to respond during a peak demand. After a DR event when the set-points are back to the normal schedule, we have observed peaks while the energy is ramping up. For buildings which have a prescribed demand limit, these peaks cause demand penalty. Addressing this issue, we propose a strategy to enable stable ramping up process

Abstract

The use of Internet and Social Media has increased exponentially during the last few years globally giving the opportunity to interact with other people and share ideas, thoughts, opinions etc. Large-scale data are shared every day through social media platforms with enormous speed and reach an incredibly huge number of people. Additionally, the possibility of writing anonymous posts and comments makes it even more easy to express and spread hate speech. Social media platforms, in order to improve the experience of their users, are trying to eliminate comments expressing hatred. In this thesis, we mainly focus on developing automated techniques to identify hate-speech. We start with Traditional techniques used for hate-speech detection, followed by advanced techniques for identifying hate-speech on social media. Lastly we attempt to identify cases where these automated systems can fail miserable and propose technique to make these systems robust. Traditionally, various social media platforms, such as Wikipedia, Facebook, YouTube etc, employ hundreds of staff members as part of their human review team which manually read every reported post and decide if it is inappropriate for the users. Nowadays, having administrators detecting which comments are offensive or relying on user reports are ineffective methods not only due to the large-scale of data produces through social media and also due to the fact that during the last years an increase of hatred is noticed in the modern societies. As of 2018, there are about 317,000 status updates on Facebook every 60 seconds making it harder than finding a needle in a haystack every 60 seconds! We commence our research work with the problem of hate-speech detection on social media. Popular techniques like handcrafted feature-based Machine Learning and rule-based approaches have their own disadvantages. The former method is hard to scale and generalize, while the latter fails to provide good results making it unreliable for use in production level systems. Motivated by the improvements in the computational resources and availability of large annotated datasets, we work towards building deep learning based approaches for hate-speech detection. We propose composite models which utilizes the capability of deep models for generating concise representations along with the power of traditional machine learning classifiers. Every word has at least one meaning when it stands alone. But the meaning can change depending on context, or even over time. A sentence full of neutral words can be hostile (“Only whites should have rights”), and a sentence packed with potentially hostile words (“F**k what, f**k whatever y’all been wearing”) can be neutral when you recognize it as a Kanye West lyric. Motivated by this idea, we then move towards building techniques that utilize the contextual information instead of using just the content for identifying hate-speech. We propose a Context-based neural network architecture that utilizes the relationship between the content & context for identifying hate-speech text. Modeling relationship between the content & the neighboring text (known as context), helps in resolving disambiguity in cases where its very difficult to identify if the text is abusive or not just from the content. Finally we identify issues with the automated hate-speech detection systems and propose novel techniques to build robust models. Use of social-media data for training automated systems poses risk of it learning biases which are abundant on social media. Almost every forum is biased in some form, either women, gender, race, right-wing groups, gay/lesbians, religion etc. Learning powerful systems that can change the user experience requires the system to be fair towards every religion, group or gender. The artificial intelligence can not yet detect/understand these lingual nuances. We propose techniques that identify a type of bias, stereotypical bias, in an arbitrary model and then propose techniques on we can learn unbiased-artificially intelligent systems by observations from the biased social media. We carry out extensive experiments to establish the effectiveness of our methods. We also perform experiments to prove the efficacy of our model on a real-world API, the Perspective API, and show its effectiveness in identifying biased systems.

Abstract

In this thesis, we propose a novel planning technique for monocular camera based Simultaneous Localization and Mapping(VSLAM) and fast Frontier detection method for autonomous explorartion in an indoor evvironment. In VSLAM, the objective is to estimate the trajectory of camera and simultaneously identify key 3D feature points and build a map, using camera as a depth sensor. Unlike a laser range finder(LRF) based SLAM, VSLAM is known to be erroneous when camera motion includes an in-place rotation or feature displacement is large for successive frames. We propose a motion planning framework which combines motion primitives based planning and trajectory optimization approach to generate trajectories which exactly connects an initial and final state and also ensures that the change in camera’s field of view between subsequent instances is less than some specified threshold. As a consequence of this motion planning framework we are able to automate SLAM and generate automated monocular SLAM maps of an indoor lab area. We also show when the robot follows the path of a generic planner, PTAM trajectory breaks more often than when it executes the path computed by the proposed planner. This performance improvement is further utilised to develop an autonomous vision based exploration system. Frontier detection is a critical component in autonomous exploration, wherein the robot decides the next best location to move in order to continue its mapping process. The existing frontier detection methods require dense reconstruction which is difficult to attain in a poorly textured indoor environment using a monocular camera. In this effort, we present an alternate method of detecting frontiers during the course of robot motion that circumvents the requirement of dense mapping. Based on the observation that frontiers typically occur around areas with sudden change in texture (zero-crossings), we propose a novel linear chain Conditional Random Field(CRF) formulation that is able to detect the presence or absence of frontier regions around such areas. We use cues like spread of 3D points and scene change around these areas as an observation to CRF. We demonstrate that this method gives us more relevant frontiers compared to other monocular camera based methods in the literature. Finally, we present results in an indoor environment, wherein frontiers are reliably detected around walls leading to new corridors, doors leading to new rooms or corridors and tables and other objects that open up to a new space in rooms.

Abstract

Sequence Variants are the most common variants observed in human genome, with single nucleotide variations (SNVs) occurring at a frequency of 1 in every 300bp and small insertions and deletions (INDELs) at a slightly lower frequency. When these variations occur in the coding regions, they can have a deleterious effect on the protein product and function, leading to disease situations. Reducing costs of next generation sequencing technologies has made it possible to take genetic profiling to the clinical setting. This requires not just fast and accurate algorithms for variant detection, but also a knowledge-base for variant annotation and prioritization to facilitate tailored therapeutics based on an individual’s genetic profile. With this objective, we have developed a pipeline, Sequence Variants Identification and Annotation (SeqVItA) that provides an end-to-end solution for small sequence variants detection, annotation and prioritization on a single platform. Parallelization of the variant detection step and incorporation of numerous resources to infer functional impact, clinical relevance and drug-variant associations, SeqVItA will benefit the clinical and research communities alike. Its open-source platform and modular framework allows for easy customization of the workflow depending on the data type (single, paired or pooled samples), variant type (germline and somatic), and variant annotation and prioritization. Performance comparison of SeqVItA on simulated data, real genomic data, and detection, interpretation and analysis of somatic variants on 24 liver cancer patients is carried out. In this study we demonstrate the efficacy of annotation module in facilitating personalized medicine based on patient’s mutational landscape.

Abstract

Science journalism is the art of conveying a detailed scientific research paper in a form that non-scientists can understand and appreciate while ensuring that its underlying information is conveyed accurately. It aims to transform jargon-laden scientific articles into a form that a common reader can comprehend while ensuring that the meaning of the article is retained. It plays a crucial role in making scientific content suitable for consumption by the public at large.Recent advances in Deep Learning research and it’s applications in natural language processing have made way to impressive results in Natural Language Generation. We leverage these advances to explore the possibility of their use in journalism, science journalism in particular, as comprehension of scientific content is much harder challenge than most of the other forms of content, like shorthand, which journalists use while writing articles. In this work, we introduce the problem of automated science journalism and present ways to automate some parts of the workflow by automatically generating the ‘title’ of a blog version of a scientific paper. We have built a corpus of 87, 328 pairs of research papers and their corresponding blogs from two science news aggregators and have used it to build Science-Blogger a pipeline based architecture consisting of a two-stage mechanism to generate the blog titles. To demonstrate the models, we built an interactive tool, where a user can give abstract and title of a research paper, which would be processed by our APIs to produce a blog title, along with some relevant information about the model used for the generation. Evaluation using standard metrics indicate viability of the proposed system. Twitter is another social platform where a lot of content of every imaginable category is shared. Naturally, it is also one of the popular media for sharing scientific work apart from blogs. So, we have also experimented with our model to generate tweets, which are roughly of similar length as blog titles. Evaluation on generated tweets also showed promising results. Although significant advances are made in Natural Language Generation, we noticed that evaluation methods didn’t catch up with these advances. Popular metrics like ROUGE and BLEU are based on word overlap. None of these metrics ensure that the generated sentences do not contradict the truth (actual content in the article). It is quite possible that the generated sentences have high overlap with human-written sentences while still contradicting the human-written sentence. Misrepresenting facts is considered a serious problem in journalism. So, a system automating it should explicitly ensure that such misrepresentation and false statements are not generated. Hence, we also formulated a metric to evaluate sentences generated by such automated abstractive natural language generation systems called SSAS: Semantic Similarity for Abstractive Summarization. Ideally a metric evaluating an abstract sys-tem summary should represent the extent to which the system-generated summary approximates the semantic inference conceived by the reader using a human-written reference summary. Most of the previous approaches relied upon word or syntactic sub-sequence overlap to evaluate system-generated summaries. Such metrics cannot evaluate the summary at semantic inference level. Through this work we introduce the metric of Semantic Similarity for Abstractive Summarization (SSAS), which leverages natural language inference and paraphrasing techniques to frame a novel approach to evaluate system summaries at semantic inference level. SSAS is based upon a weighted composition of quantities representing the level of agreement, contradiction, topical neutrality, paraphrasing, and optionally ROUGE score between a system-generated and a human-written summary.

Abstract

Automatic Speech Recognition (ASR) systems have witnessed a lot of progress in the past decade. In high resourced scenarios like English, ASR systems have shown performance compared to human parity level on specific tasks. Speech recognition systems for Indian languages are less studied compared to other high resourced languages like English. India is a developing country with large emerging markets for speech recognition technology. Developing Indian language ASR systems requires certain challenges to addressed which are innate to Indian languages. India is a multilingual society. India has 23 official languages. To penetrate deep into the Indian markets, ASR systems which can be operated in multiple languages are to be developed. Collecting data from a multilingual environment is much tedious task than to acquire data from a monolingual environment. Often Indian language ASR systems have to be developed for low-resourced scenarios. Apart from multilingual nature, bilingualism is very prevalent in the Indian population which leads frequent code-switching and word borrowing between any two languages. Operating parallel ASR systems with code-switching capabilities in Indian scenarios is a huge challenge. This motivated us to work towards multilingual ASR systems which can handle codemixing and word borrowing efficiently. In this thesis, we address various issues related to development of ASR systems for Indian scenarios. An integrated ASR system is developed using common phone-set which can efficiently handle multilingual code-mixed speech. Acoustic modeling approaches such as HMM-GMM, HMM-SGMM and RNN-CTC have been studied to find the most suitable acoustic model. Various acoustic modeling units such as context independent phone, context dependent phone and syllables are explored to find the most suitable unit for developing joint acoustic models. Residual connections have been explored to improve the performance of the joint acoustic models. Studies directed towards supplementing the conventional features along with articulatory features have been explored for developing mulit-lingual ASR systems. Fricative landmarks are detected and the detected landmarks are used as the features for improving the performance of multilingual ASR system. Distinctive features from speech are modeled using a statistical approach and their relevance for improving the performance of a multilingual ASR is explored. In a low resourced scenario when the data is pooled from multiple sources, the meta-level information about the speaker is not accessible. A speaker normalization to handle those scenarios is explored. Some major conclusions from the work are: • Using Common phone-set for training joint acoustic models offers an attractive solution developing ASR systems to handle multilingual and code-mixed scenarios. • Acoustic models that can model context independent phones have performed better compared to the context dependent tri-phone units. RNN-CTC based joint-acoustic model has performed better that HMM-SGMM model. • Using residual networks to develop the joint acoustic models have stabilized the training. They have improved the performance of the acoustic model when the model is sufficiently deep. • Fricative landmarks when fused with the features have improved the performances of the ASR systems when the data size is small, with relatively larges sized datasets the performance have not improved significantly. • Using distinctive features predicted from DNNs along with the input features have not significantly improved the performance of multilingual ASR system. • When the meta-information about the speaker ID is not available, Using the speaker codes derived from a speaker ID network can be used to improve the performance of a multilingual ASR system

Abstract

Empathy allows one to understand distinct cognitive and emotional states of another. Empathy is critical for social interactions and survival. For researchers it is an interesting but challenging proposition to study either in an observer or in the perpetrator of a crime. In the context of rape, observer’s empathic response is required to help the victim overcome the trauma and deliver justice, while lack of empathy in the perpetrator is of interest to understand deviant behavior. Empathy or the lack of it is influenced by perceptions, from self-experiences or those imposed by social stereotypes, and this is in particular evidenced in attitude towards rape victims. The ability to empathize with a rape victim or display the empathy to not commit an aggressive criminal act is based on complex processing of one’s expectations of the physical characteristics, socioeconomic status, and perceptions of credibility and hence deemed to be highly subjective. Each of societal stereotypes requires in-depth study to model expected empathic response or profile deviation. In this thesis we examine a small but critical parameterthat is the rape victim’s appearance and its relation to observer’s empathy. Our hypothesis is loosely based on observations of typical responses post-rape incidence are reported, which is that female victim carrying a non-traditional (as defined by a particular culture- that is, short hair with no bindi/dot on forehead) look, with a darker skin tone and resident of a village evokes lower empathy response in the common citizens. A set of two experiments were designed to understand behavioral (pupillary response) and the neural activations (functional magnetic resonance imaging) associated with empathy networks. In this first study, for stimuli the average female Indian computer-generated faces are photoshopped to reflect a) variation in skin tone/color, b) (non)traditional appearance and c) inclusion of place of residence(urban/rural). Prior to being shown the faces, the participants (20 male graduate students) were informed that the faces were of rape victims. The faces were designed to not show any expression, to reduce bias from facial cues. The results from a preliminary investigation, shown the presence of the motion/empathy network comprising of the insulaanterior cingulate cortex (ACC)-inferior frontal gyrus (IFG) areas to female faces of purported rape victims for all the category of faces, though higher activation in the Insula (area crucial for emotional empathy) was observed for traditional faces compared to non-traditional faces. Similarly, higher activation in empathy network was observed for fair/lighter skin tone faces compared to darker tone faces. No differential activation was found on the basis of place of residence. Interestingly, activations in the perspective or cognitive empathy regions covering the prefrontal cortex were not observed for any condition. In the second experiment, computer-generated faces were photoshopped to reflect lighter to darker skin tones, and pupil size changes as a function of reflected light were correlated to cognitive processes of ‘like’ through rating provided by the participants (70 engineering students). The hypothesis was that reflex reaction to skin tone was partly due to inadequate contrasts especially in darker skin tone before social bias of race is triggered. The results show slight differences in pupil size when perceiving a light tone face or a dark tone face but statistical significance was not found. The average values from the data indicate that people who ‘like’ dark color have smaller pupil size and larger pupil size was noticed in the group that ‘like’ lighter face tones. It is also observed that subjects who marked their self-color as the dark/medium tend to have a preference for the same skin tones in others. The results of the two independent studies provide preliminary evidence for social perceptions of physical appearance and the strength of the techniques (fMRI or eye tracking) to measure empathy and perceptions. Further studies on convicted rapists are required to understand empathy deficiency for rape victims and the underlying reasons for it.