Abstract

The ability to convey information quickly, accurately, and efficiently is one of the primary aims of human innovation. With the invention of the electronic numerical integrator and computer, an interaction between informatics and telecommunication began which added faster data processing to distant communication. The conventional circuit switching was soon replaced by packet switching, and the TCP/IP protocol suite, facilitating communication between computers/heterogeneous machines, ultimately leading to the birth of the Internet. The advent of the Internet has resulted in the redefining of traditional modes of communication like radio, television and paper mail giving rise to services such as email, digital newspapers, Internet telephony and video streaming. With networks growing exponentially, modern communication methods have become the driving force of social evolution. The ever increasing amount of information being generated presents new challenges in terms of storage, transfer and security. Since computers can range from stand-alone to networked devices, various avenues of attacks are available to compromise information security. Ensuring confidentiality of information being transmitted and stored is therefore essential. Also known as access control, this can be achieved through authentication. Authentication is the process of verifying a claimed identity. The validation method can involve multiple factors with the level of security being proportional to the number and type of factors involved. Mutual authentication is where the involved parties simultaneously authenticate each other to establish a connection. It is most often implemented machine-to-machine through digital certificates where there is a chance of the user not realizing when the remote authentication fails. Challenge-response based mechanisms help mitigate this problem by detecting false end-points. In this thesis, we study the importance of authentication and key agreement in two different network applications – Internet of Things (IoT) and Session Initiation Protocol (SIP). The first study presents a new Elliptic Curve Cryptography (ECC) signaturebased authenticated key establishment scheme for applications in IoT environment. The proposed scheme accommodates password and biometric update as well as stolen/lost smartcard revocation phases. The proposed scheme has been proved to be secure using the widely-used Burrows-Abadi-Needham logic (BAN logic), informal security analysis, and also a formal security verification using the broadly-accepted Automated Validation of Internet Security Protocols and Applications (AVISPA) tool. The practical demonstration of the scheme is evaluated using the widely-accepted NS2 simulator for various network performance parameters. Finally, it is shown that the scheme provides more functionality features, and its computational and communication costs are also comparable with other existing approaches. In the next study, we focus on authentication in Session Initiation Protocol for Voice over IP environments. In this work, we present an efficient ECC based three-factor authentication and session key agreement scheme for SIP, which uses the identity, pass- word and personal biometrics of a user as three factors. The proposed scheme resolves the shortcomings in existing SIP authentication protocols. The proposed scheme also supports password and biometric update phase without involving the server and the user mobile device revocation phase in case the mobile device is lost/stolen. Formal security analysis under the Real-Or-Random (ROR) model and the broadly-accepted BAN logic ensures that the proposed scheme can withstand several known security attacks. The proposed scheme has also been analyzed informally to show that it can also withstand other known attacks. Simulation for formal security verification using the widely-known AVISPA tool shows that the scheme is secure against replay and man-in-the-middle attacks. Finally, high security, and low communication and computation costs make the proposed scheme more suitable for practical application as compared to other existing related ECC-based schemes for SIP authentication. Keywords:

Abstract

Active learning also known as query learning is a sub-field of machine learning. It relies on the assumption that if the learning algorithm is allowed to choose the data from which it learns, it will perform better with less training. Active Learning is predominantly used in areas where getting a large amount of annotated data for training is not feasible or extremely expensive. Active learning models aims to overcome the annotation bottleneck by asking queries in the form of unlabelled instances to be labelled by a human. In this way, the framework aims to achieve high accuracy using very less labelled instances resulting in minimization of annotation cost. In the first part of our work, we propose an Active Learning based Image Annotation model. Automatic image annotation is the computer vision task of assigning a set of appropriate textual tags to a novel image. The aim is to eventually bridge the semantic gap of visual and textual representations with the help of these tags. The advantages of the proposed model includes: (a). It is able to output the variable number of tags for images which improves the accuracy. (b). It is effectively able to choose the difficult samples that needs to be manually annotated and thereby reducing the human annotation efforts. Studies on Corel and IAPR TC-12 datasets validate the effectiveness of this model. In the second part of the thesis, we propose an active learning based solution for efficient, scalable and accurate annotations of objects in video sequences. We focus on reducing the human annotation efforts with simultaneous increase in tracking accuracy to get precise, tight bounding boxes around an object of interest. We use a novel combination of two different tracking algorithms to track an object in the whole video sequence. We propose a sampling strategy to sample the most informative frame which is given for human annotation. This newly annotated frame is used to update the previous annotations. Thus, by collaborative efforts of both human and the system we obtain accurate annotations with minimal effort. We have quantitatively and qualitatively validated the results on eight different datasets. Active Learning is efficient in Natural Language documents as well. Multilingual processing tasks like statistical machine translation and cross language information retrieval rely mainly on availability of accurate parallel corpora. In the third section we propose a simple yet efficient method to generate huge amount of reasonably accurate parallel corpus using OCR with minimal user efforts. We show the performance of our proposed method on a manually aligned dataset of 300 Hindi-English sentences and 100 English-Malayalam sentences. In the last section we utilised Active Learning for model updation in c QA system. Community Question Answering(c QA ) platforms like Yahoo! Answers, Baidu Zhidao, Quora, StackOverflow etc. provides experts to give precise and targeted answers to any question posted by a user. These sites form huge repositories of information in the form of questions and answers. Retrieval of semantically relevant questions and answers from c QA forums have been an important research area for the past few years. Considering the ever growing nature of the data in c QA forums, these models cannot be kept stagnant. They need to be continuously updated so that they can adapt to the changing patterns of Questions-Answers with time. Such updation procedures are expensive and time consuming. We propose a novel Topic model based active sampler named Picky. It intelligently selects a smaller subset of the newly added Question-Answer pairs to be fed to the existing model for updating it. Evaluations on real life c QA datasets show that our approach converges at a faster rate, giving comparable performance to other baseline sampling strategies updated with data of ten times the size.

Abstract

Since the last two decades, vast amounts of digital data have been created, a large portion of which is the visual data comprising images and videos. To deal with this large amount of data, it has become necessary to build systems that can help humans to efficiently organize, index and retrieve from such data. While modern search engines are quite efficient in text-based indexing and retrieval, their “visual cortex” is still evolving. One way to address this is to enable similar technologies as those used for textual data for archiving and retrieving the visual data. Also, in practice people find it more convenient to interact with visual data using text as an interface rather than a visual interface. However, this in turn would require describing images and videos using natural text. Since it is practically infeasible to annotate these large visual collections manually, we need to develop automatic techniques for the same. In this thesis, we present our attempts towards modelling and learning semantic associations between images and different forms of text such as labels, phrases and captions. Our first problem is that of tagging images with discrete semantic labels, also called image annotation. To address this, we describe two approaches. In the first approach, we propose a novel extension of the conventional weighted k-nearest neighbour algorithm that tries to address the issues of class-imbalance and incomplete-labelling that are quite common in the image annotation task. In the second approach, we first analyze why the conventional SVM algorithm, despite its strong theoretical properties, does not achieve as good results as nearest neighbour based methods on the image annotation task. Based on this analysis, we propose an extension of SVM by introducing a tolerance parameter into the hinge-loss. This additional parameter helps in making binary models tolerant to practical challenges such as incomplete-labelling, label ambiguity and structural overlap. Next we target the problem of image captioning and caption-based image retrieval. Rather than using either individual words or entire captions, we propose to make use of textual phrases for both these tasks; e.g., “aeroplane at airport”, “person riding”, etc. These phrases are automatically extracted from available data based on linguistic constraints. To generate a caption for a new image, first the phrases present in the neighbouring (annotated) images are ranked based onvisual similarity. These are then integrated into a pre-defined template for caption generation. During caption based image retrieval, a given query is first decomposed into such phrases, and images are then ranked based on their joint relevance with these phrases. Lastly, we address the problem of cross-modal image-text retrieval. For this, we first present a novel Structural SVM based formulation for this task. We show that our formulation is generic, and can be used with a variety of loss functions as well as feature vector based representations. Next, we try to model higher-level semantics in multi/cross-modal data based on shared category information. For this, we first propose the notion of cross-specificity, and then present a generic framework based on cross-specificity that can be used as a wrapper function over several cross-modal matching approaches, and helps in boosting their performance on the cross-modal retrieval task. We evaluate the proposed methods on a number of popular and relevant datasets. On the image annotation task, we achieve near state-of-the-art results under multiple evaluation metrics. On the image captioning task, we achieve superior results compared to conventional methods that are mostly based on visual cues and corpus statistics. On the cross-modal retrieval task, both our approaches provide compelling improvements over baseline cross-modal retrieval techniques.

Abstract

Internet users today prefer getting precise answer to their questions rather than sifting through a bunch of relevant documents provided by search engines. This has led to the huge popularity of Community Question Answering (cQA) services like Yahoo! Answers, Baidu Zhidao, Quora, StackOverflow etc., where forum users respond directly to questions with short targeted answers. These forums provide a platform for interaction with experts and serve as popular and effective means of information seeking on the Web. Anyone can obtain answers to their questions by posting them for other participants on these sites. Community can also decide the quality of answers for a question. Over time, such cQA archives have become rich repositories of knowledge encoded in the form of questions and user generated answers. However, not all questions get immediate answers from other users. If a question is not interesting enough for community or if similar question is already answered by some other user, it may suffer from “starvation”. Such questions may take hours and sometimes days to get satisfactory answers. This delay in response can be avoided by searching similar questions in the very large archives of previously asked questions. If a similar question is found, then the corresponding best answer can be provided without any delay. The main challenge while retrieving similar questions is the “lexico-syntactic gap” between the user query and the questions already present in the forum. The aim is to detect question pairs that differ from each other lexically and syntactically but expresses the same meaning. In this thesis, we propose two novel approaches to bridge the lexico-syntactic gap between the question posed by the user and forum questions. In the first approach, we design “Deep Structured Topic Model (DSTM)” which retrieves similar questions that lie in the vicinity of the latent topic vector space of the query and the archived question-answer pairs. The retrieved topically similar questions are reranked using a deep semantic model. In the second approach, we explore the behaviour of deep semantic models with “parameter-sharing” between the parallel networks which help us to design “Siamese Convolutional Neural Network for cQA (SCQA)”. It consists of twin convolutional neural networks with shared parameters and a contrastive loss function joining them. It learns the similarity metric for question-question pairs by leveraging the question-answer pairs available in cQA forum archives. The model projects semantically similar question pairs nearer to each other and dissimilar question pairs farther away from each other in the semantic space. Several models have been built in the past to bridge the lexico-syntactic gap in the cQA content. However, considering the ever growing nature of the data in cQA forums, these models cannot be kept stagnant. They need to be continuously updated so that they can adapt to the changing patterns of questions-answers with time. Such updation procedures are expensive and time consuming. In this thesis, we propose a novel Topic model based active sampler named Picky. It intelligently selects a smaller subset of the newly added question-answer pairs to be fed to the existing model for updating it. Experiments on large scale real-life “Yahoo! Answers” dataset reveals that DSTM and SCQA outperforms current state-of-the-art approaches based on translation models, topic models and existing deep neural network based models. Also, evaluations on real life c QA datasets show that “Picky” converges at a faster rate, giving comparable performance to other baseline sampling strategies updated with data of ten times the size.

Abstract

Age related macula degeneration (AMD), Cystoid Macular Edema (CME) and glaucoma are retinal diseases that affects vision and often lead to irreversible blindness. Many imaging modalities have been used for screening of retinal diseases. Optical coherence tomography (OCT) is one such imaging modality that provides structural information of retina. Optical coherence tomography angiography (OCTA) also provides vascular information of retina in addition to structural information. With advancement in OCT and OCTA technology, the number of patients being scanned using these modalities is increasing exponentially. Manual analysis of vast data often makes human experts feel fatigue. Also, this extends the time to treat any patient and creates a demand to develop a fast and accurate automated OCT image analysis system. The system proposed in this thesis aims at analysing retinal anatomy and its diseases. We approach the problem of automatically analysing the retinal anatomy by segmenting its layers using a deep learning framework. In literature these algorithms usually requires preprocessing steps, such as denoising, image flatenning and edge detection, all of which involve separate parameter tunings. We propose a deep learning technique to automate all these steps and handle the presence/absence of pathologies. This model consists of a combination of Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM). The CNN is used to extract layers of interest image and extract the edges, while the LSTM is used to trace the layer boundary. This model is trained on a mixture of normal and AMD cases using minimal data. Validation results on three public datasets show that the pixel-wise mean absolute error obtained with our system is 1.30 ± 0.48 which is lower than inter-marker error of 1.79 ± 0.76. The performance of the proposed module is also on par with the existing methods. We next propose three modules for disease analysis, first for diagnosing them at volume level, second at slice level and finally at pixel level by segmenting the abnormality. Firstly, we propose a novel method for glaucoma assessment at volume level using data from OCTA modality. The goal of this module is to gain insights about glaucoma indicators from both structural information (OCT volume) and vascular information (angioflow images). The proposed method achieves a mean sensitivity of 94% specificity of 91% and accuracy of 92% on 49 normal and 18 glaucomatous volumes. Secondly, for slice level disease(AMD and CME) classification from OCT volumes, we learn a decision boundary using a CNN in the new extremal representation space ofan image. Evaluating on four publically available datasets with training set consisting of 3500 OCT slices and test set having 1995 OCT slices, this module achieves a mean sensitivity of 96%. specificity of 97% and accuracy of 96%. Finally, we propose an automated cyst segmentation algorithm from OCT volumes. We propose a biologically inspired method based on selective enhancement of the cysts, by inducing motion to a given OCT slice. A CNN is designed to learn a mapping function that combines the result of multiple such motions to produce a probability map for cyst locations in a given slice. The final segmentation of cysts is obtained via a simple clustering of the detected cyst locations. The proposed method is trained on OPTIMA Cyst segmentation challenge (OCSC) train test and achieves a mean Dice Coefficient (DC) of 0.71, 0.69 and 0.79 on the OCSC test set, DME dataset and AEI dataset respectively. Thus, the complete system can be employed in screening scenarios to aid retinal experts in accurate and faster analysis of retinal anatomy and disease.

Abstract

With overwhelming amount of visual data on the internet, it is beyond doubt that a search capability for this data is needed. While several commercial systems have been built to retrieve images and videos using meta-data, many of the images and videos do not have a detailed descriptions. This problem has been addressed by content based image retrieval (CBIR) systems which retrieve the images by their visual content. In this thesis, we will demonstrate that images and videos can be retrieved using the pose of the humans present in them. Here pose is the 2D/3D spatial arrangement of anatomical body parts like arms and legs. Pose is an important information which conveys action, gesture and the mood of the person. Retrieving humans using pose has commercial implications in domains such as dance (query being a dance pose) and sports (query being a shot). In this thesis, we propose three pose representations that can be used for retrieval. Using one of these representations, we will build a real-time pose retrieval system over million movie frames. Our first pose representation is based on the output of human pose estimation algorithms (HPE) [5, 26, 80, 103] which estimate the pose of the person given an image. Unfortunately, these algorithms are not entirely reliable and often make mistakes. We solve this problem by proposing an evaluator that predicts if a HPE algorithm has succeeded. We describe the stages required to learn and test an evaluator, including the use of an annotated ground truth dataset for training and testing the evaluator, and the development of auxiliary features that have not been used by the (HPE) algorithm, but can be learnt by the evaluator to predict if the output is correct or not. To demonstrate our ideas, we build an evaluator for each of four recently developed HPE algorithms using their publicly available implementations: Andriluka et al. [5], Eichner and Ferrari [26], Sapp et al. [80], and Yang and Ramanan [103]. We demonstrate that in each case our evaluator is able to predict whether the algorithm has correctly estimated the pose or not. In this context we also provide a new dataset of annotated stickmen. Further, we propose innovative ways in which a pose evaluator can be used. Specifically, we show how a pose evaluator can be used to filter incorrect pose estimates, to fuse outputs from different HPE algorithms, and to improve a pose search application. Our second pose representation is inspired by poselets [13] which are body detectors. First, we introduce deep poselets for pose-sensitive detection of various body parts, that are built on convolutional neural network (CNN) features. These deep poselets significantly outperform previous instantiations of Berkeley poselets [13]. Second, using these detector responses, we construct a pose representation that is suitable for pose search, and show that pose retrieval performance is on par with the state of the artpose representations. The compared methods include Bag of visual words [85], Berkeley poselets [13] and Human pose estimation algorithms [103, 18, 68]. All the methods are quantitatively evaluated on a large dataset of images built from a number of standard benchmarks together with frames from Hollywood movies. Our third pose representations is based on embedding an image into a lower dimensional pose-sensitive manifold. Here we make the following contributions: (a) We design an optimized neural network which maps the input image to a very low dimensional space where similar poses are close by and dissimilar poses are farther away, and (b) We show that pose retrieval system using these low dimensional representation is on par with the deep poselet representation. Finally, we describe a method for real time video retrieval where the task is to match the 2D human pose of a query. A user can form a query by (i) interactively controlling a stickman on a web based GUI, (ii) uploading an image of the desired pose, or (iii) using the Kinect and acting out the query himself. The method is scalable and is applied to a dataset of 22 movies totaling more than three million frames. The real time performance is achieved by searching for approximate nearest neighbors to the query using a random forest of K-D trees. Apart from the query modalities, we introduce two other areas of novelty. First, we show that pose retrieval can proceed using a low dimensional representation. Second, we show that the precision of the results can be improved substantially by combining the outputs of independent human pose estimation algorithms. The performance of the system is assessed quantitatively over a range of pose queries.

Abstract

Diabetic Retinopathy (DR) is a condition where individuals with diabetes develop a disease in the inner wall of eye known as retina. DR is a major cause of visual impairments and early detection can prevent vision loss. Use of automatic systems for DR diagnosis is limited due to their lower accuracy. As an alternative, reading-centers are becoming popular in real-world scenarios. Reading center is a facility where retinal images coming from various sources are stored and trained personals(who might not be experts) analyze them. In this thesis we look at techniques to increase efficiency of DR image-readers working in reading centers. The first half of this thesis aims at identifying efficient image-reading technique which is both fast and accurate. Towards this end we have conducted an eye-tracking study with medical experts while they were reading images for DR diagnosis. The analysis shows that experts employ mainly two types of reading strategies: dwelling and tracing. Dwelling strategy appears to be accurate and faster than tracing strategy. Eye movements of all the experts are combined in a novel way to extract an optimal image scanning strategy, which can be recommended to image-readers for efficient diagnosis. In order to increase the efficiency further, we propose a technique where saliency of lesions can be boosted for better visibility of lesions. This is named as an Assistive Lesion Emphasis System(ALES) and demonstrated in the second half of the thesis. ALES is developed as a two stage system: saliency detection and lesion emphasis. Two biologically inspired saliency models, which mimic human visual system, are designed using unsupervised and supervised techniques. Unsupervised saliency model is inspired from human visual system and achieved 10% higher recall than other existing saliency models when compared with average gazemap of 15 retinal experts. Supervised saliency model developed as deep learning based implementation of biologically inspired saliency model proposed by Itti-Koch(Itti, L., Koch, C. and Niebur, E., 1998) and achieves 10% to 20% higher AUC compared to existing saliency model when compared with manual markings. Saliency maps generated by these models are used to boost the prominence of lesions locally. This is done using two types of selective enhancement techniques. One technique uses multiscale fusion of saliency map with original image and other uses spatially varying gamma correction; both increases CNR of lesions by 30%. One saliency model and one selective-enhancement technique are clubbed together to illustrate two complete ALESs. DR diagnosis done by analyzing ALES output using optimal strategy should presumably be faster and accurate.

Abstract

In most of the applications, wireless sensor networks (WSNs) are deployed in an unattended environment. Thus, sensor nodes can be physically captured by the adversaries, and the adversaries can then extract the stored information in those nodes using the power analysis attacks. Using the extracted information they can manufacture new sensor nodes. These newly manufactured sensor nodes can also have additional functionalities, which can be used to launch different attacks. After that these new sensor nodes can be deployed by the adversaries in the network. These nodes can launch different attacks in the network, which can cause information loss along with high energy expenditure. Furthermore, WSN is prone to various attacks, such as blackhole attack, wormhole attack, sinkhole attack, etc. In a blackhole attack, an attacker can physically capture and re-program a set of nodes in the network to block the packets they receive instead of forwarding them towards the BS. Any information that enter in the blackhole region is then compromised, and the information does not reach to the destination. This produces high end-to-end delay, and decrease in packet delivery ratio and network throughput. As a result, the information cannot reach to the destination node within the required time period. In a sinkhole attack, an attacker node first advertises a best possible route (with less hop-distance route) to the destination (BS) to attract its neighbors so that they may fall into this attraction to utilize the advertised route more frequently. The neighbors can then forward their traffic through the efficient advertised route declared by the sinkhole attacker node. The route can also captivate other nodes apart from the neighbor nodes of the sinkhole attacker node, which are closer to the sinkhole than the BS. So, the attacker node has an opportunity to tamper with the data, damage the regular network operations or conduct other serious threats. Sometimes multiple attacks can also be launched in the network using the hybrid anomaly. Hybrid anomaly degrades the network performance rapidly and also trouble the attack specific detection mechanisms. In this situation, it is very crucial to find out what kind of anomaly is activated in the network. In the presence of blackhole attack, sinkhole attack and hybrid anomaly, the confidential information can be delayed, leaked, altered or even can not reach to the destination. Therefore, it becomes essential to provide robust and efficient solutions for the detection and prevention of blackhole attack, sinkhole attack and hybrid anomaly in HWSN. This motivates us to propose new detection and prevention protocols for these types of attacks in HWSN. In this thesis, we propose several intrusion detection and prevention schemes in hierarchical wireless sensor networks (HWSNs). The first contribution is on designing a new efficient group-based scheme for the detection and prevention of multiple blackhole attacker nodes in HWSNs. In our scheme, the entire WSN is divided into several clusters, and each cluster has a resource-rich cluster head (CH) node. Each CH is responsible for the detection and prevention of blackhole attack in the network. For the detection purpose, each CH uses the proposed detection algorithm by which it sends status-cum-data query messages to all its cluster members (sensor nodes). CH then waits for the reply to those messages from its members. Node status messages (e.g. sleeping, working, idle) are merged with data query messages in order to further reduce the total communication cost of the proposed scheme. After the detection of blackhole nodes, CH nodes use the proposed responsibility reassignment algorithm in order to reassign the responsibilities of the blacklisted nodes (blackhole attacker nodes) to the normal nodes. Our scheme is secure against blackhole attacker nodes, which is shown through the analytical and the simulation results using the widely-accepted NS2 simulator. The proposed scheme achieves about 90% detection rate and 3.75% false positive rate, which are significantly better than the existing related schemes. Furthermore, our scheme is efficient, and thus, it is very appropriate for practical applications in HWSNs. In the second contribution, we propose a new detection scheme for the detection of different types of sinkhole nodes in HWSNs. To the best of our knowledge, this is the first attempt to design such a detection scheme in HWSNs which can detect sinkhole message modification nodes (SM D), sinkhole message dropping nodes (SDP ) and sinkhole message delay nodes (SDL) nodes. In our approach, the entire HWSN is divided into several disjoint clusters, and each cluster has a resource-rich high-end sensor node (called a cluster head), which is responsible for the detection of different sinkhole attacker nodes if present in that cluster. Our scheme is secure against sinkhole attacker nodes, which is shown through the ana

Abstract

In remote sensing, multi-sensor data fusion aims at combining strengths of different data sources in a manner such that the resultant fused product preserves the best of the participating sources. From among all the levels at which fusion is done, namely pixel level, feature level and decision/information level, pixel level data fusion is the most elementary form of fusion. The real advantage with pixel level fusion is that it has the actual raw input data at its disposal which could be transferred to the output in a form closest to the input. Hence, pixel level algorithms need to aim at preserving the input properties rather than just improving the overall visual quality. Most of the existing methods are aimed at visual quality improvement and perform fusion using the global distributions, which results in loss of uniqueness of certain objects features. Such objects are either completely lost or their responses are milder in the output. These kind of local distributions can be preserved using a localized method of fusion. Other limitations of fusion methods include distortion of overall spectral distribution of the input data source and their applicability on the type of data being fused. There is a Gaussian Linear Stretch Image (GLSI) technique [1] for images fusion to handle spectral distortion. It is a moving window-based distribution matching and details are transferred in a linear manner. We have enhanced this algorithm to effectively handle two major challenges it faced. One is the detection and matching of class distributions which are inverted in the low and high resolution images and the other being the fusion of pixels belong to class distributions with significantly low sample space. We call it the Enhanced Gaussian Linear Stretch Image (e-GLSI) algorithm. The proposed e-GLSI algorithm is also a window based technique which aims at improving spatial resolution of multispectral bands using higher resolution panchromatic band. e-GLSI fuses the two inputs by transferring the spatial details of the panchromatic band into each multispectral band independently at local window level, hence prevents suppression of local unique features and preserves the spectral distribution of each band as well. In this approach, the local window statistics are used to establish linear relationships between corresponding windows from two different spatial resolution bands. Cases of distribution inversions and unimodal splits are handled using well defined metrics before fusing two distributions together. The algorithm then enhances each band independently without using any derived or aggregated knowledge, thus making this technique suitable to fuse any combination of input images. Results show that the statistical profile of the e-GLSI output bands matches the original multispectral image bands’ statistical profile better than other widely used approaches. In order to evaluate this matching quantitatively, standard deviation of the relative difference in fused pixel value and original multispectral pixel value per band has been computed. e-GLSI output gives a standard deviation of 0.18, 0.12 and 0.26 for red, blue and green bands respectively. Thus showing that it’s able to preserve the spectral characteristics from input images. In Spite of being able to preserve the spectral characteristics of the input images, there are some artifacts introduced in the e-GLSI fusion output. One of them is objects getting split across the moving window of the technique, which introduces a window boundary or sharp edge in some of the objects. A way to handle this could be the use of a variable sized window depending on the area of the image being processed. But this in itself is a challenge as it would require surrounding information and certain heuristics in order to decide the window size at a location in an image. In order to tackle this challenge, we used objects as windows in our approach. We propose an object based fusion algorithm based on GLSI, which we call the Object based GLSI (o-GLSI). Processing blocks for this approach are are the objects segmented from the Pan image. Since pixels being fused are part of a single object, we transfer details in a Gaussian manner in case of o-GLSI. The quantitative evaluation of o-GLSI fusion results clearly show that o-GLSI standard deviation of 0.32, 0.49 and 0.82 for relative difference in fused pixel and ground truth multispectral pixel for red, blue and green bands respectively is successful in keeping the fusion output spectrally closer to the input. Universal Image Quality Index (UIQI) [2] values of 0.9 for each band with respect to the ground truth proves that the quality of image fused by o-GLSI is very good. Since classification is one of the prominent applications for fused images, a superior accuracy rate supports the effectiveness of the fusion algorithm involved. As another measure to evaluate the utility of the fused images, k-means classification accuracy has been co

Abstract

Empathy entails the ability to understand (perspective-taking) and share the affective experiences of others, both processes considered the bedrock of all social interactions and critical for survival. Most empathy related neuroscience studies employ simple paradigms like pain-infliction or static images with emotional context and analysis focuses on either perspective-taking processing or affective experience overlooking the complex interaction among the processes. The initial studies informed how one understands pieces of the whole system with an assumption that when information from the pieces is fit the whole picture can be realized but this methodology has led to constrained theories on empathy process. To address the short-comings in understanding empathy, the focus of affective neuroscience research has shifted to studying this phenomenon as part of naturalistic social cognition. Hence with the goal to mimic real-life situations that evoke empathy, naturalistic or ecologically-valid paradigms are recent inclusions building upon the foundation provided by the earlier simple models. This study focuses specifically on narratives and empathy response. The aim of this study is to explore the neural correlates of narrative empathy using multi-modal visual movie narrative. Towards this, two functional magnetic resonance imaging (fMRI) studies were conducted using a total of 5 commercial full-length short film and movie clips. The first experiment had 3 movie clips (5-8minute long) from a diverse genre (animation, Hollywood and Indian Hindi movie) and the goal of this exploratory investigation was to identify the networks of empathy – cognitive (mentalizing), emotional and motor (affective experiences) and analyzing the dynamic nature of the neural activations as a function of the self-reported level of empathy experienced. The data-driven independent component analysis (ICA) method applied to isolate the underlying brain networks revealed: a) the ability of the fictional narratives induces emotional contagion and the potential to elicit empathy response b) activation of significant, distinct but overlapping empathy networks and c) the dynamic cross-correlation analysis between the task default-mode network (DMN) and individual empathy networks revealed that emotional empathy response displayed lower correlation while cognitive empathy higher correlation with DMN, suggesting the possible role of self-reflection or attention to an internal narrative triggered by external stimuli (Andrews-Hanna et al., 2012; Gusnard et al., 2001). This is a potentially significant finding for understanding the difference in cognitive and emotional empathy in clinical conditions like autism and psychopathy. Developing on the findings from the first study, a second fMRI study was designed to understand specifically the role of context in empathy response. Toward this, two diverse full-length short movies (8 minutes each) with real-life actors of ethnicity dissimilar to the participants were selected as stimuli. Functional imaging data was collected from two sets of extremely short clips (13-20seconds) with neutral and emotional scenes extracted from the full-length movie followed by the whole movie. Using a general linear model approach for data analysis, the contrasts estimated from each condition (activation for the short clips of neutral and emotional scenes versus the same clips in the full-length movie) were compared and the analysis showed that empathy sub-processes (emotional, cognitive and motor) were statistically significant when context is presented. The results also reveal the complex differential empathy processes in real-life social interactions, where responses are a function of context. Overall, the neural correlates of narrative empathy presented in this thesis, the first of the kind, suggest that a complex construct such as empathy should be best studied in with ecologically valid stimuli. The impact of the results, in studying empathy deficiency and interventions using narratives to enhance empathy, needs to be explored in the future.