Abstract

Semiconductor Industry has undergone rapid miniaturization in recent decades due to advances in manufacturing technology and the demand for more functionality per chip area. However, scaling down metal-oxide-semiconductor field-effect transistors (MOSFETs) offer many advantages in terms of functionality and speed; it also poses many challenges like increased leakage power. A shrink in device size leads to lowering the threshold voltage to meet the functionality metrics. Due to the inverse relationship of leakage power with the threshold voltage, a rapid increase in leakage power is encountered. In deep sub-micron technologies, static power is dominant over dynamic power, leading to an urgent need for leakage reduction techniques. The primary outcome of scaling down transistors is a reduction in propagation delay. This is achieved by the lowering of threshold voltage and oxide thickness. However, as this comes with the higher cost of increased sub-threshold leakage current, scaling down of Vt is restricted. Hence, the need for delay optimization techniques. Another major issue associated with the miniaturization of MOSFETs is the variation of process parameters. Due to scaled-down devices, slight variation in process parameters impacts significantly on performance and yield of the devices. In addition, external environmental changes like temperature and supply voltage variations also affect the performance parameters like delay and leakage by affecting device densities and switching activities of various blocks. Hence, Process variations, Voltage and Temperature (PVT) are the key factors to be considered while optimizing a circuit. Furthermore, aging due to Negative-bias temperature instability (NBTI) affects the delay of the circuit. So, NBTI aging should be considered in delay optimization, for manufacturing more reliable and long-life devices. The above factors call for the indispensable need for leakage-delay optimization techniques. Hence, in this research, we propose a multi-objective algorithm-based leakage-delay optimization approach. We used various intuitive multi-objective algorithms to model device parameters such as the Width (W) and Length (L) of a MOSFET. Considering PVT variations for low-power optimization and Process, Supply Voltage, Temperature and Aging (PVTA) variations for high-performance applications, we have applied various swarm-based and genetic algorithms on basic and complex benchmark circuits and obtained up to 70% optimization of Leakage power with PVT and 30% optimization of propagation delays with PVTA conditions. Further, we developed an automated framework of algorithms to analyze the characteristics of a given circuit, choose the appropriate algorithm and load-based sizing to replace in a given circuit for obtaining the required performance metrics. In addition, we have proposed Backward traversal replacement and Partitioning large cells methods for enhanced optimisation of complex cells. The next step to further optimize this optimization process of the circuits is to eliminate the inherent extensive run time taken by the SPICE tool. The algorithm optimization process involves the calculation of circuit performances like leakage and propagation delay for huge samples. moreover, with the scaled down technology nodes, statistical variation aware analysis has gained importance, which also involves large number of circuit performance calculations. SPICE tool is widely used to calculate the circuit performances and computing results of an extensive sample set with the SPICE tool is time complex. Hence, we proposed a statistical variation aware estimation models based on machine learning techniques like Polynomial Regression and Artificial Neural Networks (ANN), which replaces the SPICE tool with <1% Error and 10,000 times reduction in time complexity.

Abstract

There is no doubt that videos are today’s most popular content consumption method. With the rise of the streaming giants such as YouTube, Netflix, etc., video content is accessible to more people. Naturally, video content creation has also increased to cater to the rising demand. In order to reach out to a wider audience, the creators dub their content. An important aspect of dubbing is not only changing the speech but also lip synchronizing the speaker in the video. Talking-face video generation works have achieved state-of-the-art results in synthesizing videos with accurate lip synchronization. However, most of the previous works deal with low-resolution talking-face videos (up to 256 × 256 pixels), thus, generating extremely high-resolution videos still remains a challenge. Also, with advancements in internet and camera tech more and more number of people are able to create video content and that too in ultra high resolution such as 4K (3840 × 2160). In this thesis, we take a giant leap and propose a novel method to synthesize talking-face videos at resolutions as high as 4K! Our task presents several key challenges: (i) Scaling the existing methods to such high resolutions is resource-constrained, both in terms of compute and the availability of very high-resolution datasets, (ii) The synthesized videos need to be spatially and temporally coherent. The sheer number of pixels that the model needs to generate while maintaining the temporal consistency at the video level makes this task non-trivial and has never been attempted in literature. We propose to train the lip-sync generator in a compact Vector Quantized (VQ) space for the first time to address these issues. Our core idea to encode the faces in a compact 16 × 16 representation allows us to model high-resolution videos. In our framework, we learn the lip movements in the quantized space on the newly collected 4K Talking Faces (4KTF) dataset. Our approach is speaker agnostic and can handle various languages and voices. We benchmark our technique against several competitive works and show that we can achieve a remarkable 64-times more pixels than the current state-of-the-art! Now, how to edit videos using the above algorithm or any other deep learning algorithm? To do so, the person has to download the source code of the required method and run the code manually. How amazing would it be if people could use the deep learning techniques in video editors with a click of a single button? In this thesis, we also propose a video editor based on OpenShot with several state-of-theart facial video editing algorithms as added functionalities. Our editor provides an easy-to-use interface to apply modern lip-syncing algorithms interactively. Apart from lip-syncing, the editor also uses audio and facial re-enactment to generate expressive talking faces. The manual control improves the overall experience of video editing without missing out on the benefits of modern synthetic video generation algorithms. This control enables us to lip-sync complex dubbed movie scenes, interviews, television shows, and other visual content. Furthermore, our editor provides features that automatically translate lectures from spoken content, lip-sync of the professor, and background content like slides. While doing so, we also tackle the critical aspect of synchronizing background content with the translated speech. We qualitatively evaluate the usefulness of the proposed editor by conducting human evaluations. Our evaluations show a clear improvement in the efficiency of using human editors and an improved video generation quality

Abstract

The latest digital revolution has caused an outburst of data. With growing data in different online platforms, it is crucial to process and streamline the data into valuable information. Extensive research is being carried out to achieve the same goal, and for achieving this, we understand that Natural Language Understanding (NLU) plays a vital role. It is considered a core component of many applications like Dialogue Systems, Question Answering, Text Classification, Automatic Text Generation, etc. However, as of today, a lot of the research in this direction is limited to the English language only. Our main aim is to extend these to other resource-poor languages like Telugu. Hence we propose a Dialogue System for Telugu that internally does Question Classification and Named Entity Recognition (NER). NER is one of the essential parts of NLU. It helps us in extracting essential phrases present in the sentence. Once we identify the essential phrases, we can classify them as various entity types and then use this information for decision-making. Our attempt is one step closer to bridging the gap between user and computer interaction in Telugu by proposing a sustainable method for creating domain-specific dialogue systems with limited data. We also improve the availability of various methods and tools that facilitate Natural Language Understanding and research of NLU in Telugu by building the state-of-the-art Named Entity Recognition for Telugu and providing sufficient annotated datasets. As discussed above, we first propose a dialogue system for the Hospital domain in Telugu. The main aim of the dialogue system is communication with the user in spoken or written form, and they have been gaining popularity over the past decade since they are found useful and helpful for the users. The dialogue system for the Hospital domain handles various hospital and doctor-related queries. However, the idea is to present an approach for modeling a dialogue system in a resource-poor language by combining linguistic and domain knowledge. Focusing on the question-answering aspect of the dialogue system, we identified Question Classification and Query Processing as the most vital parts of this dialogue system. Our method combines deep learning techniques for question classification and rule-based computational analysis for query processing. Human evaluation of the system has been performed as there is no automated evaluation tool for dialogue systems in Telugu. The results show that our system achieves a high overall rating and a significantly accurate context-capturing method. On the other hand, Named Entity Recognition (NER), which facilitates applications like dialogue systems, is about identifying entities in a sentence. It is a successful and well-researched problem in English due to the availability of resources. The transformer models, specifically the masked-language models (MLM), have shown remarkable performance in NER during recent times. However, with grow ing data and applications in all languages, there is a need for NER in other languages too. But, NER remains to be explored in Indian languages due to the lack of resources and tools. Our contributions include (i) Three annotated NER datasets for the Telugu language in multiple domains: Newswire Dataset (ND), Medical Dataset (MD), and Combined Dataset (CD). (ii) Comparison of the finetuned Telugu pretrained transformer models (BERT-Te, RoBERTa-Te, and ELECTRA-Te) with other baseline models (CRF, LSTM-CRF, and BiLSTM-CRF). (iii) Further investigation of the performance of Telugu pretrained transformer models against the multilingual models mBERT [20], XLM-R [15], and IndicBERT [45]. We find that pretrained Telugu language models (BERT-Te and RoBERTa) outperform the existing pretrained multilingual and baseline models in NER. On a large dataset (CD) of 38,363 sentences, the BERT-Te achieves a high F1-score of 0.80 (entity-level) and 0.75 (token-level). Further, these pretrained Telugu models have shown state-of-the-art performance on various existing Telugu NER datasets. We open-source our dataset, pretrained models, and code

Abstract

Globally, due to the changes in climate, rapid urbanization, land use changes, and increased water demands, water quantity-quality management has become a significant sustainability problem. Maintaining a complete natural river flow is usually impossible due to the development of the water resources and variations in land and soil usage in the catchment. These developed resources can create differences in the balance of the ecosystem and socioeconomic activities. Stream flow is a prime factor affecting the river flow health and quality in such watersheds. Admitting many issues associated with the water quality of the rivers, stream flow plays a vital role as the changes in the stream flow can directly transmit the changes in water quality. The variations in the flow regimes over time relatively affect the riverine ecosystems of the watersheds. Environmental Flows (EFs) assessment, which depends on the flow regime, illustrates the quantity and quality of water flow required for the sustainability of freshwater, estuarine ecosystem, and human livelihood. In this context, Environmental Flow Allocations (EFA) of reservoir-river systems do not adequately address the connections between reservoir inflows, releases, and corresponding downstream river water quality, rendering water resources management difficult. Thus, this study aims to develop an integrated, holistic modeling approach for Eflow estimation by combining a hydrological model, reservoir release analysis, Global Environmental Flow Calculator (GEFC), and river water quality model. In this present study, the GEFC method is used to estimate the ecological flows by using the flow duration curves (FDC) generated from the given monthly discharge data of the river. The FDC in this system contains 17 fixed percentile points concerning the discharge. In the current study, we analyze the Tunga-Bhadra River basin's Environmental flows (Eflows) by considering different discharge stations. The stations are Balehonuur, Haralahalli, Hosaritti, Shivamogga, Honalli, Rattihalli, and Tunga-Bhadra Dam, with a mean annual flow (MAF) of 36%, 24.8%, 27.2%, 16.2%, 23.3%, 21.1%, and 12.2% respectively to maintain the ecological conditions of the river. The monthly discharge data from 1995 to 2017 for those stations are obtained from the Advance Centre for Integrated Water Resource (ACIWR) in Bengaluru, India. River flow health is a study that helps understand the environmental variables that affect the river's habitat structure, flow regime, water quality, and biological conditions. To estimate the flow health of the Tunga-Bhadra River, we used a tool called Flow Health which uses nine indicators to represent the Flow Health (FH) score for the stations Balehonnur, Haralahalli, Hosaritti, Shivamogga, Honalli, Rattihalli, and Tunga-Bhadra. This tool uses the gauge discharge data in the form of reference (1995-2005) and test periods (2006-2017), with Flow Health score of 0.72, 0.4, 0.72, 0.70, 0.58, 0.73, 0.71 and 0.72, 0.63, 0.63, 0.7, 0.66, 0.67, 0.66 for test and reference period with respect to stations. The study noted that most of the discharge stations along the Tunga-Bhadra River show moderate to low flow variations for the reference and test periods. Overall, Tunga-Bhadra river health, measured by the flow indices, declined from 1995-2005 to 2006-2017. A hydrological model, Soil & Water Assessment Tool (SWAT), is selected to simulate the inflows of the Bhadra reservoir. The calibration and validation of the SWAT estimated inflows are done using the SWAT-CUP tool, which shows satisfactory results. The SWAT findings were utilized in reservoir release analysis to determine the reservoir releases. Environmental Flow Management Classes (EMCs) for estimating and assessing the minimum flow to be maintained in a river were derived using the GEFC for the Bhadra watershed. The river water quality-modeling tool, QUAL2K, was chosen to simulate the Dissolved Oxygen (DO) corresponding to the EMCs developed from the GEFC for the Bhadra Reservoir. The study considered various plausible scenarios of inflows (10 to 20% reduction), pollution scenarios of Biological Oxygen Demand (BOD) (0 to 100 % treatment) for arriving drains by altering headwaters DO (4 to 8 mg/l). The study revealed that by maintaining the headwater DO (about 7 mg/l) in conjunction with BOD treatment of drains (25%) and by maintaining sufficient Eflow allocation (class D with 1269 Million Cubic meters (MCM)); downstream river water quality could be improved (6.5 mg/l of DO at Holehonnur monitoring point of Bhadra reservoir-river system, India). From all these performed analyses, we provided an insight into the reservoir supply management and maintaining the riverine ecosystem by combining the water quantity and quality models corresponding to the environmental aspects.

Abstract

Navigation of Unmanned Aerial Vehicles (UAVs) amongst high-rises in urban environments be-comes inevitable due to increasing demand for various applications in Urban Air Mobility (UAM).Moreover, these city scale urban environments are populated with tall buildings and skyscrapers withinherent piecewise planar structures. Leveraging this inherent structural information for navigating aUAV equipped with a single camera and an IMU is the primary focus of this thesis. We make threecontributions that leverage these geometric cues to detect and map these planar structures for obstacleavoidance and path planning.Our first contribution,Multi-View Planarity Constraints for skyline estimation from UAV im-ages in city scale urban environments, is a three-stage pipeline that brings together several modulescombining a data-driven monocular plane segmentation network and geometric constraints from 3D vi-sion to showcase a quick reconstruction of planar facades within a few views. We evaluate the efficacyof our pipeline with various constraints and errors from multi-view geometry using ablation studies. Wethen retrieve the skyline of the buildings in synthetic as well as real-world scenes.In our second contribution,A new geometric approach for three view line reconstruction andmotion estimation in Manhattan Scenes, we propose a novel method of pose estimation using linefeatures from three views of a Manhattan Scene. We leverage the vanishing point directions to estimatethe relative rotations as well as to fix the 3D line direction. In consequence we build a constraints matrix,which has the relative translations and 3D line depth as its null space. We then perform 1-parameter lineBA using factor graph based cost function. We compare the efficacy of our method with standard linetriangulation in synthetic as well as real-world scenes.Finally, we proposeUrbanFly: Uncertainty-Aware Planning for Navigation Amongst High-Rises with Monocular Visual-Inertial SLAM Maps, a sequential convex program based novel trajec-tory planner tailored for outdoor urban scenes. It uses the sparse point clouds generated by a MonocularVisual Inertial SLAM (VINS) backend to build a cuboid representation of the environment through adata-driven monocular plane segmentation network. Our chosen world model provides faster distancequeries than the more common voxel-grid representation. The trajectory optimizer is initialized based onsafety estimates on a set of randomly drawn trajectories. We perform extensive simulations in a tightlyintegrated perception-control loop to validate its efficacy. UrbanFly outperforms competing baselinesin metrics such as collision rate, trajectory length, etc., on a high fidelity AirSim simulator augmentedwith synthetic and real-world dataset scenes.

Abstract

The current research presents an experimental investigation on coal ash samples (fly ash, bottom ash, and pond ash) collected from the NALCO thermal power plant and Talcher Thermal Power station, Odisha, India, and a numerical investigation on the dynamic response of shallow foundation on soil and pond ash deposit. Morphological, mineralogical, and geotechnical properties of coal ash (fly ash, bottom ash, and pond ash) samples were extensively investigated. The pond ash sample was chosen for further study because of its unique gradation of all collected samples. It was intended to determine the dynamic characteristics of pond ash subjected to dynamic excitation through laboratory tests and numerical analysis. A set of multiple cyclic triaxial tests were planned and performed on pond ash to examine the influence of test parameters like relative compaction, effective confining pressure, frequency, and shear strain. The properties obtained through the experimental tests were applied in the numerical analysis on pond ash subjected to different seismic excitations, i.e., the Nepal earthquake (Mw: 7.8) and Northeast India earthquake (Mw: 7.5). Liquefaction-induced deformation of pond ash was evaluated from the numerical analysis. Initially, morphological, mineralogical, and geotechnical properties of coal ash (fly ash, bottom ash, and pond ash) were investigated. Then pond ash was subjected to dynamic loads using cyclic triaxial testing equipment. The factors influencing the liquefaction potential of pond ash were evaluated. At low frequency (0.3 Hz), the pond ash specimen takes 87 cycles to liquefy at its most densified state. Also, the maximum dynamic shear modulus was noticed at this stage, i.e., 6534.80 kPa, and the decay was observed at 87 cycles. In addition, the decay in dynamic shear modulus was rapid with the enhancement of cyclic shear strain. The presence of 50% fines and the highly compacted state of the pond ash specimen caused close packing of particles which offered more resistance to shear strain; this resulted in possession of the high shear modulus as observed in this study. The maximum value of the damping ratio was obtained for the highly compacted specimen confined at high pressure subjected to the high frequency at medium cyclic shear strain considered in this study. The amplification of shear strain from medium to large shear strain caused a decrement in the damping ratio of pond ash. The presence of 50% fines in the present pond ash specimen made the specimen exhibit less dynamic shear modulus (Gdyn) (i.e., 33% less) than that of past studies by Mohanty and Patra (2014). The presence of predominant silt range particles (51%) made the specimen offer less resistance against the applied high strains despite better interlocking of silt and sand range particles of compacted pond ash specimens in the present study. 50% degradation of the dynamic shear modulus occurred within 4-28 cycles of loading. The dissipation of energy over the first few loading cycles was noticed between 12% and 94%, and it continued with a decreasing trend afterward. The observed values of the damping ratio were in contrast with that of the past studies on pond ash tested for similar adopted parameters (Mohanty and Patra, 2014) and contrary to the studies on different kinds of sandy soils (Kokusho, 1980; Govindarajulu, 2005). Studies by Chattaraj and Sengupta (2016) on fly ash with a predominant range of silt particles (~80%) revealed that the damping ratio was increased with an increase in shear strain; this observation contrasts with the present study. The trend of a damping ratio beyond 0.5% cyclic shear strain is in good agreement with the past studies of Kumar et al. (2017). A regression analysis has been carried out for the energy dissipated during cyclic loading under a varying state of compaction, effective confining pressure, and frequency. Few relationships between energy dissipated during cyclic loading under varying parameters of the state of compaction, effective confining pressure, and frequency were generated. The percentage variation of observed and predicted results is 4%. The 1D, 2D, and 3D ground response analysis of soil and pond ash and seismic response analysis of soil and pond ash with the foundation were subjected to Nepal and Northeast India earthquake excitations. The size of strip footing and shallow foundation (square) for 2D and 3D response analysis has been considered 1.5m x 0.8m and 1.5m x 1.5m x 0.8m, respectively. The PGA value in the case of 1D ground response analysis of pond ash has been noticed as 1.24 times and 1.63 times over 2D and 3D ground response analysis, respectively, when excited under the Northeast India earthquake. The PGA value in the case of 1D ground response analysis of pond ash has been noticed as 0.73 times and nearly two times over 2D and 3D ground response analysis, respectively, when excited under the Nepal earthquake. Soil and pond ash were found to be liq

Abstract

Geometric approach to solving a problem seems very natural or sometimes elegant compared to an analytical solution. Several machine learning problems have hidden geometric structures that can be exploited in proposing new solutions. In this work, we consider a specific structure called Riemannian structure of the search space of the problem. Often such spaces are nonlinear spaces where the standard techniques of optimization cannot work for the lack of linearity. But these spaces have a very useful property of being locally linear, which in turn helps us to consider extensions of classical optimization to such spaces. In particular, we consider two machine learning problems - the extreme classification problem and the tensor completion problem, where we show the presence of Riemannian structure. Once we identify this structure, we show how to use it to come up with efficient algorithms. For the extreme classification problem, a proof the convergence is given for the proposed algorithm. We experimentally verify the efficiency of the method in terms of train time and test accuracy comparing with several state-of-the-art methods on extremely large datasets. In the case of tensor completion problem, we propose a general problem that considers several works in the literature as its special case. For this general problem we propose a Riemannian optimization algorithm that solves the problem efficiently. We verify the correctness of the proposed algorithm both theoretically and experimentally through the special cases.

Abstract

Natural Language Generation is the task of producing understandable human text from variety of input sources. With the advent of pretrained language models (PLMs), text generation capabilities of current systems have achieved unprecedented heights. Pretrained language models have become the new normal and naturally serve as a backbone architecture for numerous tasks. It has been observed that these models learn various intricacies involved in language understanding and generation during pretraining step. These models are pretrained over large corpus; they also discover the world knowledge through text, some of which are absorbed in model parameters. Although, during finetuning on certain knowledge-intensive tasks (like text coherence, data-to-text, summarization, translation, etc.), it fails to utilize the intrinsic knowledge stored in it’s parameters effectively. In this thesis, we will tackle this problem by incorporating external facts to improve results on two downstream tasks: multilingual facts-to-text generation and text coherence modeling. We observe close association of facts in improving text generation by directly focusing on fact-to-text generation task. The fact-to-text generation is a variant of data-to-text where structured input data is knowledge graph triples. Data-to-text generative system consumes structured input data like tables, databases, knowledge bases, time-series data etc., and produces human-readable text summaries. The first part of the thesis addresses the multilingual fact-to-text generation, where facts are used to generate sentences in multiple languages. The fact-to-text generation requires a dataset where knowledge graph triples are well-aligned with semantically equivalent textual information. Manual creation of such a highquality fact-to-text dataset requires human supervision and is quite challenging to scale. Unsupervised alignment has recently emerged as an active area of research to overcome lack of labelled data and difficulty in domain adaptation. However, not much work has been done for low-resource languages that provide two significant challenges: (1) unavailability of pair of triples and native text with same content distribution and (2) limited Natural language Processing resources in low-resource languages. Hence, we rigorously investigate cross-lingual fact-to-text problem of aligning English structured data with sentences in multiple low-resource languages and develop a new dataset called XALIGN consisting of 0.45M pairs across seven low-resource languages. We propose two different methods of cross-lingual fact-to-text alignment: (a) Non-parametric approaches and (b) Parametric approaches. Additionally, we experimented with strong baseline results by adapting popular natural language generation methods for the cross-lingual fact-to-text task An essential requirement for any system that generates text is the coherence. In the second part of thesis, we address detection of text coherence. A large body of previous work have leveraged entity-based methods, syntactic patterns, discourse relations, and traditional deep learning architectures for text coherence assessment. However, these approaches do not consider factual information present in the documents. Transitions of facts associated with entities across sentences could help better capture the essence of textual coherence. We hypothesise that coherence assessment is a cognitively complex task that requires deeper fact-aware models and can benefit from other related tasks. To demonstrate this, we develop a novel deep learning model that fuses document-level information with factual information. We further enhance the model efficacy by training it simultaneously with Natural Language Inference tasks in multi-task learning setting, taking advantage of inductive transfer between the two tasks. Our experiments with popular benchmark datasets across multiple domains demonstrate that the proposed model consistently outperforms existing methods on synthetic coherence evaluation tasks and two real-world tasks involving predicting varying degrees of coherence.

Abstract

In times where autonomous navigation has the center stage, it is important for the embodied agents to learn to navigate through the environment with the degree of flexibility, that resembles the human-like sensibility and decision-making capacity. To do so, it is important to understand the spatial and semantic correlation between the regions and the objects and encapsulate this knowledge into the agent. In this thesis, we contribute to the same by proposing frameworks which utilize the relational co-occurrence of different regions and the objects within them in the problem of ”Object Goal Navigation”. The Object Goal Navigation (ObjectNav) task requires a robot to move to an instance of an out of view target object class, when initialised at a random position, in an unseen environment. The first framework proposed, uses a history of robot trajectories to learn a Spatial Relational Graph (SRG) and Graph Convolutional Network (GCN)-based embeddings for the likelihood of proximity of different semantically-labeled regions and the occurrence of different object classes in these regions. When the robot has to locate a specific target object instance during evaluation, a Bayesian inference approach and the SRG are used to estimate the visible regions, and the learned GCN embeddings are used to rank different visible regions and select the region to explore next. The approach is tested using the Matterport3D (MP3D) benchmark dataset of indoor scenes in AI Habitat, a visually realistic simulation environment. We report a improvement in performance over relevant baselines. The second framework described, first builds a semantic map of the environment at gradually over time, and then repeatedly selects a long-term goal based on the semantic map to locate the target object instance. The decision about ”where to go” (long-term goal) is formulated as a vision-based deep reinforcement learning problem. Specifically, an Encoder Network is trained to process a semantic map, extract high-level features, and select a long-term goal. In addition, we incorporate data augmentation and Q-function regularization to make the long-term goal selection process more effective. We report experimental results using the photo-realistic Gibson benchmark dataset in the AI Habitat 3D simulation environment to demonstrate that our framework substantially improves performance on standard measures in comparison with a state of the art baseline

Abstract

Speech is the most convinient and expressive form of communication among humans. Since the past few decades there have been continous efforts from the technology community around the world to establish speech based human-computer interaction. In this respect, several speech based technologies including (but not limited to) automatic speech recognition, text to speech synthesis, speaker recognition, speaker verification and emotion recognition have evolved. Among the various natural signals that can be recorded using sensors (such as seismic signal, radio signals etc.), we have physiological understanding of the corresponding production process of only a few signals such as speech and cardiac signal. In this thesis, we explore the possibility of extracting sound information from speech using it’s corresponding production characteristics. In particular, certain acoustic features are proposed for the identification of stops and nasals in continous speech. These acoustic features extract the corresponding sound’s production characteristics at two levels, namely, excitation source level and vocal tract system level. The acoustic features are extracted only at glottal closure instants using recently proposed signal processing techniques. The proposed algorithms do not rely on training on a database, and hence can be used for analysis on under-resourced languages. Further another study is presented to assess the individual contributions of magnitude and phase spectra of a given speech signal in preserving it’s intelligibility under different noise conditions. The results in this study can be used to develop algorithms for modifying the speech signal such that it’s intelligibility is improved in noise.