Dissertations / Theses on the topic 'Test automation framework analysis'

The validation approach for assuring quality of software does often include the conduction of tests. Software testing includes a wide range of methodology depending on the system level and the component under test. Graphical user interface (GUI) testing consists of high level tests that assert that functions and design element in user interfaces work as expected. The research conducted in this paper focused on GUI testing of web based applications and the movement towards automated testing within the software industry. The question which formed the basis for the study was the following: How should a generic test automation framework be designed in order to allow maintenance between developers and non-developers? The study was conducted on a Swedish consultant company that provides e-commerce web solutions. A work strategy approach for automated testing was identified and an automation framework prototype was produced. The framework was evaluated through a pilot study where testers participated through the creation of a test suite for a specific GUI testing area. Time estimations were collected as well as qualitative measurements through a follow up survey. This paper presents a work strategy proposal for automated tests together with description of the framework system design. The results are presented with a subsequent discussion about the benefits and complexity of creating and introducing automated tests within large scale systems. Future work suggestions are also addressed together with accountancy of the frameworks usefulness for other testing areas besides GUI testing.

Nesse trabalho foi criado um framework para a realização de testes automatizados de acordo com as práticas de desenvolvimento dirigido por testes, o qual pode ser estendido com novas linguagens de especificação, utilizando idéias de desenvolvimento dirigido por comportamentos. Espera-se que, ao utilizar esse framework, equipes de projetos de software possam especificar testes em uma linguagem mais adequada para o domínio de sua aplicação, através do suporte de uma ferramenta que permita tal nível de personalização. O framework foi instanciado com a criação de uma linguagem baseada em máquinas de estado, utilizada para a aplicação da ferramenta em um projeto real. O objetivo deste trabalho é avaliar os impactos da aplicação dessas técnicas, através da experiência de utilização da ferramenta no projeto. A motivação para esse trabalho surgiu da experiência no desenvolvimento de sistemas de controle, usualmente reativos e com requisitos de tempo real, nos quais muitas vezes a descrição de seus comportamentos é feita através de máquinas de estado, que possivelmente é a melhor linguagem para este domínio.<br>In this work we create a framework for automated testing according to test driven development practices that can be extended to support new specification languages, following ideas from behavior driven development. We expect that, by using this framework, software development teams will be able to specify tests in more proper languages for the application domain, with the support of a tool that allows such level of customization. The framework was instantiated with the creation of a language based on state machines, and used on a real-life project. The goal of this work is that software project teams become able to specify tests in the most fitting language for their application domain, through the support of a tool that makes possible such level of customization. The motivation for this work comes from the experience with the development of control systems, usually with real-time requirements, which often have their behaviors described by state machines, possibly the best language for this domain.

In the modern wireless communication industry, radio communication equipment vendors not only produce communication hardware, but also produce software. In fact, software revenue is now a large part of the total revenue. As technology has developed and traffic demands increase, more and more functions required to implement the radio system are implemented via software rather than hardware. Today, many hardware functions are actually implemented with reconfigurable and programmable hardware. Therefore, it is often possible to perform an upgrade by loading new software (a software upgrade) rather than needing to change the physical hardware with every technology advance. However, introducing new elements and features in existing (often mature) software may cause unexpected problems. These problems may include new parts malfunctioning and failure or degradation of old functions. To avoid these problems, each version of software has to be thoroughly tested, not only to test the new parts, but also to verify that the old functions still work properly. Testing all the old functions is time and human resource consuming. Thus, there is an increasing demand for automated testing. This thesis will focus on why automated regression testing is necessary and how to implement automated testing in a specific environment. The thesis results show that automated testing can improve the test coverage by at least 40% for one of Ericsson’s WCDMA software releases. This coupled with a reduction in testing time enables more rapid development by significantly reducing the test time without compromising quality. All of these results lead to improved profitability and increased customer satisfaction.<br>I den moderna trådlösa kommunikationen industrin, radioutrustning leverantörerna inte bara producera kommunikation hårdvara, utan också producera mjukvara. Faktum är programvara inkomster är nu en stor del av de totala inkomsterna. Eftersom tekniken har utvecklat och trafik krav ökar, fler och fler funktioner som krävs att genomföra radiosystem genomförs via mjukvara istället för maskinvara. Många hårdvara fungerar faktiskt genomförs med omkonfigurerbara och programmerbar hårdvara. Därför är det ofta möjligt att utföra en uppgradering av lastning ny programvara (en mjukvaruversionen) snarare än behöver för att ändra den fysiska hårdvaran med varje teknik förväg. Men att införa nya element och funktioner i befintliga (ofta äldre) programvara kan orsaka oväntade problem. Dessa problem kan innehålla nya delar brister och fel eller försämring av gamla funktioner. För att undvika dessa problem, varje version av programvaran måste testas, inte bara att testa de nya delarna, men även för att kontrollera att de gamla funktionerna fortfarande arbete ordentligt. Testa alla gamla funktioner konsumera tid och personal. Således finns det en ökad efterfrågan på automatiserade tester. Den här avhandlingen kommer att fokusera på varför automatiserad regression testning krävs och hur man genomföra automatiserade tester i en viss miljö. Avhandlingen visar att automatiserade tester kan förbättra testbunt täckning med minst 40% för ett av Ericssons WCDMA programversionerna. Detta i kombination med en minskning av provning tid möjliggör en snabbare utveckling av avsevärt minska test tid utan att kompromissa med kvaliteten. Alla dessa resultat leda till bättre lönsamhet och ökat kundvärde belåtenhet.

Context. Performance testing, among other types of non-functional testing, is necessary to assess software quality. Most often, manual approach is employed to test a system for its performance. This approach has several setbacks. The existing body of knowledge lacks empirical evidence on automation of non-functional testing and is largely focused on functional testing. Objectives. The objective of the present study is to evaluate a test framework that automates performance testing. A large-scale distributed project is selected as the context to achieve this objective. The rationale for choosing such a project is that the proposed test framework was designed with an intention to adapt and tailor according to any project’s characteristics. Methods. An experiment was conducted with 15 participants at Ericsson R&amp;D department, India to evaluate an automated test framework. Repeated measures design with counter balancing method was used to understand the accuracy and time taken while using the test framework. To assess the ease-of-use of the proposed framework, a questionnaire was distributed among the experiment participants. Statistical techniques were used to accept or reject the hypothesis. The data analysis was performed using Microsoft Excel. Results. It is observed that the automated test framework is superior to the traditional manual approach. There is a significant reduction in the average time taken to run a test case. Further, the number of errors resulting in a typical testing process is minimized. Also, the time spent by a tester during the actual test is phenomenally reduced while using the automated approach. Finally, as perceived by software testers, the automated approach is easier to use when compared to the manual test approach. Conclusions. It can be concluded that automation of non-functional testing will result in overall reduction in project costs and improves quality of software tested. This will address important performance aspects such as system availability, durability and uptime. It was observed that it is not sufficient if the software meets the functional requirements, but is also necessary to conform to the non-functional requirements.

Big amounts of test data generated by the current used software testing tools (QA-C/QA-C++ and Cantata) contain a variety of different values. The variances cause enormous challenges in data aggregation and interpretation that directly affect generation of test metrics. Due to the circumstance of data processing, this master thesis introduces a database-centric test management framework for test metrics aims at centrally handling the big data as well as facilitating the generation of test metrics. Each test result will be individually parsed to be a particular format before being stored in a centralized database. A friendly front-end user interface is connected and synchronized with the database that allows authorized users to interact with the stored data. With a granularity tracking mechanism, any stored data will be systematically located and programmatically interpreted by a test metrics generator to create various kinds of high-quality test metrics. The automatization of the framework is driven by Jenkins CI to automatically and periodically performing the sequential operations. The technology greatly and effectively optimizes and reduces effort in the development, as well as enhance the performance of the software testing processes. In this research, the framework is only started at managing the testing processes on software-unit level. However, because of the independence of the database from levels of software testing, it could also be expanded to support software development at any level.

Software Quality is a key concern for any companies working with software development. This is true due to the fact that the success of any software directly depends on Quality of software. It is expected that the software is of best quality for a long duration of time. With the introduction of Mobile applications the task of maintaining the quality of an application has been difficult and have faced many challenges. Many companies working with mobile application have reformed their process in order to maintain the quality of their application. The introduction of Automation testing in the test process is one such reform that have changed the face of mobile application testing in today’s world. This work deals with the concepts of Automation System testing for the mobile application which is until now a new thing and it has many things yet to be explored. The approach to automation testing is simple yet unique for the department of PT-MT/Quality Management in Robert Bosch GmbH based in Leinfelden, Stuttgart. Over here a selection and implementation of a test framework will be done for Automation testing of the mobile Applications that are being developed. For this a requirement specification document is being created which will form the basis for selecting a framework from the KT Analysis table. Finally, a framework TestComplete will be implemented for the already developed application "PLR measure&go" The implementation will include all the procedure required to set up the test framework as a part of documentation. The framework TestComplete will be used to create System test for iOS and Android operation system. Lastly the execution of test and the Result reporting is being shown as a complete process for Automation testing.

Questo lavoro di tesi magistrale tratta dello sviluppo di un framework per la creazione e l'automatizzazione di test su dispositivi con sistema operativo Android, descrivendone le funzionalità, i relativi dettagli implementativi e le motivazioni che hanno portato alla sua realizzazione. L'applicativo in questione, accessibile tramite una comoda interfaccia web, permette di sviluppare ed eseguire suite di test, rendendo automatici i processi di preparazione dei terminali, configurazione della macchina host e raccolta dei risultati. Funzionalità di supporto come il display interattivo in tempo reale e l'interprete dei comandi remoto agevolano il monitoraggio dei dispositivi e l'analisi di eventuali test aventi esito negativo.

The aim of this Master Thesis Frameworks for automatic testing is to compare frameworks of five most popular programming languages of today. The comparison is divided into two parts. The first one deals with comparing by using the test patterns and the second part deals with measuring of exact values while working with tools. The outcome of this thesis are criteria that can be used for comparison of other frameworks for automatic testing other than used in this work.

The train represents a complex system, where every sub-system has an important role. If a subsystem doesn’t work how it should, the correctness of whole the train can be uncertain. To ensure that system works properly, we should test each sub-system individually and integrate them together in the whole system. Each of these subsystems consists of the different modules with different functionalities what should be tested. Testing of different functionalities often requires a different approach. For some functionalities, it is necessary domain knowledge from the human expert, such as classification of signals in different use cases in Propulsion and Controls (PPC) in Bombardier Transportation. Due to this reason, we need to simulate of using experts knowledge in the certain domain. We are investigating the use of machine learning techniques for solving this cases and creating system what will automatically classify different signals using the previous human knowledge. This case study is conducted in Bombardier Transportation (BT), Västerås in departments Train Control Management System (TCMS) and Propulsion and Controls (PPC), where data is collected, analyzed and evaluated. We proposed a method for solving the oracle problem based on machine learning approach for different for certain use case. Also, we explained different steps what can be used for solving the test oracle problem where signals are part of verdict process

Submitted by Pedro Barros (pedro.silvabarros@ufpe.br) on 2018-08-01T20:03:01Z No. of bitstreams: 2 license_rdf: 811 bytes, checksum: e39d27027a6cc9cb039ad269a5db8e34 (MD5) DISSERTAÇÃO Filipe Marques Chaves Arruda.pdf: 2317425 bytes, checksum: 0739c60d4cc06fe57fcf2d3611f976cc (MD5)<br>Approved for entry into archive by Alice Araujo (alice.caraujo@ufpe.br) on 2018-08-02T20:13:13Z (GMT) No. of bitstreams: 2 license_rdf: 811 bytes, checksum: e39d27027a6cc9cb039ad269a5db8e34 (MD5) DISSERTAÇÃO Filipe Marques Chaves Arruda.pdf: 2317425 bytes, checksum: 0739c60d4cc06fe57fcf2d3611f976cc (MD5)<br>Made available in DSpace on 2018-08-02T20:13:13Z (GMT). No. of bitstreams: 2 license_rdf: 811 bytes, checksum: e39d27027a6cc9cb039ad269a5db8e34 (MD5) DISSERTAÇÃO Filipe Marques Chaves Arruda.pdf: 2317425 bytes, checksum: 0739c60d4cc06fe57fcf2d3611f976cc (MD5) Previous issue date: 2017-03-31<br>CNPq<br>A common scenario when automating tests begins with a test engineer writing test cases with no formal specification or software supervision. Then, there is a developer that tries to understand what an ambiguous test means in order to transform it into code. Finally, there is an experienced tester that is responsible to supervise the execution and to verify whether the failures are indeed bugs, or a mistake from the developer or test engineer, or just a matter of change in the requirements or user interface. To reduce these recurring problems in test automation, we propose a unified process, to write test cases using a controlled natural language to check consistency and dependencies automatically or search for similar test descriptions written in free natural language. The proposed process is applied to a reusable capture & replay strategy, so anyone can automate tests even without previous knowledge about coding, besides mitigating scalability/maintainability issues by reusing actions with granularity ranging from simple commands, such as open an application, to entire test cases or even test suites. The actions are represented by an abstract, framework-free notation. Besides, the implementation covers plugins, including voice recording and image processing; and a proactive traceability for elements in the user interface. The strategy was consolidated through its implementation in a tool, in the context of a partnership with Motorola Mobility. It has been adopted in practice for different test suites, achieving a reuse ratio up to 71% for test cases written with no standard whatsoever, yet presenting time gains similar to traditional C&R approaches when compared to coding. Furthermore, through standardization, it was possible to define a consistency notion and to capture an association graph (whose edges represent dependency and cancellation relations) among test actions. From these associations, it was possible to establish a mechanism to ensure that each test action is preceded by actions on which it depends to be granted to execute. An editor (Kaki) was implemented to mechanize both adherence to the CNL and the dependencies between tests. From the associations informed within the user interface, Kaki generates an Alloy model automatically and uses Alloy Analyzer to verify consistency, besides possibly suggesting the insertion of test actions to satisfy the dependencies, automatically.<br>Um cenário comum na automação de testes inicia-se a partir da escrita dos casos de testes por um engenheiro de testes, sem alguma especificação formal ou supervisão por software. Então, um desenvolvedor tenta interpretar o que o teste potencialmente ambíguo significa antes de transformá-lo em código executável. Finalmente, um testador é responsável por verificar se as falhas são realmente bugs, ou erros cometidos pelo desenvolvedor, pelo engenheiro de teste ou apenas uma mudança nos requisitos ou na interface de usuário. Para reduzir estes problemas recorrentes na automação de testes, nós propomos um processo unificado de escrita de casos de testes usando linguagem natural controlada que permite a verificação automática de consistência e dependências na escrita de ações de teste, ou usando uma busca por descrições similares de ações quando escritas em linguagem natural livre. O processo proposto é aplicado em uma estratégia de capture & replay reusável, assim permitindo a automação por pessoas sem background em programação, além de mitigar problemas intrínsecos de escalabilidade e manutenabilidade através do reuso de ações de teste com granularidade que pode variar desde comandos simples, como abrir um aplicativo, um caso de teste inteiro, ou mesmo uma suite de testes. As ações são representadas através de uma notação abstrata e framework-agnóstica. Além disso, a implementação ainda abrange plugins que incluem captura de voz e processamento de imagem; e rastreabilidade proativa para os artefatos. A estratégia foi concretizada através de sua implementação em uma ferramenta (Zygon) e avaliada no contexto de uma parceria com a Motorola Mobility, tendo sido adotada na prática em diferentes suítes de testes usando a linguagem natural livre, atingindo um reuso de até 71% com ganho de tempo similar a abordagens de capture & replay quando comparadas à programação. Além disso, através da padronização da linguagem natural controlada, foi possível definir uma noção de consistência e capturar um grafo de associações (dependências e cancelamentos) entre ações de teste. A partir destas associações, foi possível estabelecer mecanismos para garantir que cada ação de um teste devesse ser precedida por ações das quais a mesma depende para ser executada. Um editor (Kaki) foi implementado para mecanizar tanto aderência à CNL, como a dependência entre ações de teste. A partir das associações informadas na interface gráfica, Kaki cria automaticamente um modelo Alloy e usa o Alloy Analyzer para fazer a verificação de consistência, além de possivelmente sugerir a inserção de ações de teste para satisfazer as relações de dependências, automaticamente.

For many companies today, it is highly valuable to collect and analyse data in order to support decision making and functions of various sorts. However, this kind of data-driven approach is seldomly applied to software testing and there is often a lack of verification that the testing performed is relevant to how the system under test is used. Therefore, the aim of this thesis is to investigate the possibility of introducing a data-driven approach to test automation by extracting user behaviour data and curating it to form input for testing. A prestudy was initially conducted in order to collect and assess different data sources for augmenting the testing. After suitable data sources were identified, the required data, including data about user activity in the system, was extracted. This data was then processed and three prototypes where built on top of this data. The first prototype augments the model-based testing by automatically creating models of the most common user behaviour by utilising data mining algorithms. The second prototype tests the most frequent occurring client actions. The last prototype visualises which features of the system are not covered by automated regression testing. The data extracted and analysed in this thesis facilitates the understanding of the behaviour of the users in the system under test. The three prototypes implemented with this data as their foundation can be used to assist other testing methods by visualising test coverage and executing regression tests.

The IEEE P1687 (IJTAG) standard proposal aims at providing a standardized interface between on-chip embedded test, debug and monitoring logic (instruments), such as scan-chains and temperature sensors, and the Test Access Port of IEEE Standard 1149.1 mainly used for board test. A key feature in P1687 is to include Segment Insertion Bits (SIBs) in the scan path. SIBs make it possible to construct a multitude of different P1687 networks for the same set of instruments, and provide flexibility in test scheduling. The work presented in this thesis consists of two parts. In the first part, analysis regarding test application time is given for P1687 networks while making use of two test schedule types, namely concurrent and sequential test scheduling. Furthermore, formulas and novel algorithms are presented to compute the test time for a given P1687 network and a given schedule type. The algorithms are implemented and employed in extensive experiments on realistic industrial designs. In the second part, design of IEEE P1687 networks is studied. Designing the P1687 network that results in the least test application time for a given set of instruments, is a time-consuming task in the absence of automatic design tools. In this thesis work, novel algorithms are presented for automated design of P1687 networks which are optimized with respect to test application time and the required number of SIBs. The algorithms are implemented and demonstrated in experiments on industrial SOCs.

<p>Thorough testing of software is necessary to assure the quality of a product before it is released. The testing process requires substantial resources in software development. Model-based software development provides new possibilities to automate parts of the testing process. By automating tests, valuable time can be saved. This thesis focuses on different ways to utilize models for automatic generation of test vectors and how test coverage analysis can be used to assure the quality of a test suite or to find "dead code" in a model. Different test-automation techniques have been investigated and applied to a model of an adaptive cruise control system (ACC) used at Scania. Source code has been generated automatically from the model, model coverage and code coverage has therefore been compared. The work with this thesis resulted in a new method to create test vectors for models based on a combinatorial test technique.</p>

IEC 61850 standards are the global standard for communication in substations. It is gaining popularity in power substation automation and will dominate the future substation automation and protection system design. The standards provide new approaches for protection, control and metering function via communication. The secondary circuits in substation are simplified significantly and the massive hardwired cables are replaced by the high-speed process bus which transmit analogue and binary signals with Ethernet messages. However, the conformity of a device with the standards does not necessarily guarantee the interoperability with devices from different manufacturers. The use of devices compatible with IEC 61850 standards presents a challenge to many system integrators, especially due to lack of familiarity with features such as Generic Object Oriented Substation Event (GOOSE), reporting, Sampled Values and the need for system redundancy. To facilitate a smooth implementation, all the features and the data exchanges between devices need to be tested to ensure the system operates correctly. This project was carried out to study the protection performance of secondary schemes with IEC 61850 process bus architecture in substation. The tests were performed including current differential protection and distance protection on a transmission substation of the type used in the UK on the transmission network. The protection schemes were tested under IEC 61850 environment with multi-vendor IEDs like Alstom MiCOM IEDs, NARI IEDs with the OMICRON test set. More tests were carried out to verify the interoperability and the performance of time-critical messages were evaluated under different network architectures. The impact of the background traffic on these two messages was investigated and the response of the IEDs when the Sampled Values packets were lost or overwritten was recorded and provided to utility as a reference. This project also presented a technique to assess the performance of Merging Units from different manufacturers, when operating with Intelligent Electronic Devices (IEDs) performing a distance protection function. The performance of the process bus with parallel redundancy protocol is evaluated using a closed loop approach involving a Real Time Digital Simulator. The results indicate that protection using process bus communication is feasible, and meets grid code requirements when implemented with commercially available products. It was found that process bus operation is robust, even with network traffic conditions far beyond what would be experienced in an active substation.

The purpose of this thesis is to investigate organizational quality assurance through the international testing standard ISO 29119. The focus will be on how an organization carries out testing processes and designs and implements test cases. Keyword driven testing is a test composition concept in ISO 29119 and suitable for automation. This thesis will answer how keyword driven testing can facilitate the development of maintainable test cases and support test automation in an agile organization. The methodology used was a qualitative case study including semi-structured interviews and focus groups with agile business units within Scania IT. Among the interview participants were developers, test engineers, scrum masters and a unit manager. The results describe testing practices carried out in several agile business units, maintainability issues with test automation and general ideas of how test automation should be approached. Common issues with test automation were test cases failing due to changed test inputs, inexperience with test automation frameworks and lack of resources due to project release cycle. This thesis concludes that keyword driven testing has the potential of solving several maintainability issues with test cases breaking. However, the practicality and effectiveness of said potential remain unanswered. Moreover, successfully developing an automated keyword driven testing framework requires integration with existing test automation tools and considering the agile organizational circumstances.

Business Process Management (BPM) is a relatively new development paradigm that takes a high-level approach to coding by leveraging a graphical, “flow chart” aesthetic that allows users to assemble modular tasks into a larger process. The resulting diagrams effectively enshrine the organizations processes into an executable model that provides an objective and transparent view of the process and the activities contained within. In doing so, BPM models serve as both a system to guide employees through proper business procedures as well as documentation of the businesses processes. Similar to most other software development environments, BPM development platforms possess several features intended to address the needs of code versioning, dependency management and impact assessment. However, due to the unique way that development is done in BPM platforms, the more traditional functionality of these features sometimes renders them ineffective and ill suited to the task of BPM development. Changes to lower level reusable components in these BPM models can result in impacts to diverse processes across an organization that are difficult to predict and onerous to locate. There is much room for improvement in BPM development tools. This thesis proposes a new framework for dependency management and impact assessment to improve the usability, effectiveness and efficiency. The framework is composed of a Business Process Component Architecture, a Dependency Data Model and an Upgrade Algorithm which are all used to provide increased visibility over dependent processes and superior guidance during upgrade operations. Several example case scenarios are be used to evaluate our proposed framework. The cases represent progressive degrees of complexity to test the capabilities and robustness of the framework. Overall, the framework was able to appropriately handle the case examples used and showed promise in terms of providing practical effort, time and cost savings for BPM developers. The framework can also provide developers assistance in locating circular dependencies, but is subject to the same limitations as developers when attempting to upgrade these relationships.

A smart grid cost/benefit analysis answers a series of economic questions that address the incremental benefits of each stage or decision point. Each stage of the economic analysis provides information about the incremental benefits of that stage with respect to the previous stage. With this approach stages that provide little or no economic benefits can be identified. In this study there are series of applications,-including quasi-steady state power flows over time-varying loads and costs of service, Monte Carlo simulations, reconfiguration for restoration, and coordinated control - that are used to evaluate the cost-benefits of a series of smart grid investments. In the electric power system planning process, engineers seek to identify the most cost-effective means of serving the load within reliability and power quality criteria. In order to accurately assess the cost of a given project, the feeder losses must be calculated. In the past, the feeder losses were estimated based upon the peak load and a calculated load factor for the year. The cost of these losses would then be calculated based upon an expected, fixed per-kWh generation cost. This dissertation presents a more accurate means of calculating the cost of losses, using hourly feeder load information and time-varying electric energy cost data. The work here attempts to quantify the improvement in high accuracy and presents an example where the economic evaluation of a planning project requires the more accurate loss calculation. Smart grid investments can also affect response to equipment failures where there are two types of responses to consider -blue-sky day and storm. Storm response and power restoration can be very expensive for electric utilities. The deployment of automated switches can benefit the utility by decreasing storm restoration hours. The automated switches also improve system reliably by decreasing customer interruption duration. In this dissertation a Monte Carlo simulation is used to mimic storm equipment failure events, followed by reconfiguration for restoration and power flow evaluations. The Monte Carlo simulation is driven by actual storm statistics taken from 89 different storms, where equipment failure rates are time varying. The customer outage status and durations are examined. Changes in reliability for the system with and without automated switching devices are investigated. Time varying coordinated control of Conservation Voltage Reduction (CVR) is implemented. The coordinated control runs in the control center and makes use of measurements from throughout the system to determine control settings that move the system toward optimum performance as the load varies. The coordinated control provides set points to local controllers. A major difference between the coordinated control and local control is the set points provided by the coordinated control are time varying. Reduction of energy and losses of coordinated control are compared with local control. Also eliminating low voltage problems with coordinated control are addressed. An overall economic study is implemented in the final stage of the work. A series of five evaluations of the economic benefits of smart grid automation investments are investigated. Here benefits that can be quantified in terms of dollar savings are considered here referred to as "hard dollar" benefits. Smart Grid investment evaluations to be considered include investments in improved efficiency, more cost effective use of existing system capacity with automated switches, and coordinated control of capacitor banks and voltage regulators. These Smart Grid evaluations are sequentially ordered, resulting in a series of incremental hard dollar benefits. Hard dollar benefits come from improved efficiency, delaying large capital equipment investments, shortened storm restoration times, and reduced customer energy use. The evaluation shows that when time varying loads are considered in the design, investments in automation can improve performance and significantly lower costs resulting in "hard dollar" savings.<br>Ph. D.

<p>In this research paper, it evaluates the benefits achievable from static testing framework by analyzing and transforming the <em>JUnit3.8 </em>source code and static execution of transformed code. Static structure enables us to analyze the code statically during creation and execution of test cases. The concept of research is by now well established in static analysis and testing development. The research approach is also increasingly affecting the static testing process and such research oriented work has proved particularly valuable for those of us who want to understand the reflective behavior of <em>JUnit3.8 Framework</em>.</p><p><em> JUnit3.8 Framework</em> uses <em>Java Reflection API</em> to invoke core functionality (test cases creation and execution) dynamically. However, <em>Java Reflection API</em> allows developers to access and modify structure and behavior of a program.  Reflection provides flexible solution for creating test cases and controlling the execution of test cases. Java reflection helps to encapsulate test cases in a single object representing the test suite. It also helps to associate each test method with a test object. Where reflection is a powerful tool to perform potential operations, on the other hand, it limits static analysis. Static analysis tools often cannot work effectively with reflection.</p><p>In order to avoid the reflection, <em>Static Testing Framework</em> provides a static platform to analyze the <em>JUnit3.8</em> source code and transform it into non-reflective version that emulates the dynamic behavior of <em>JUnit3.8</em>. The transformed source code has possible leverage to replace reflection with static code and does same things in an execution environment of <em>Static Testing Framework</em> that reflection does in <em>JUnit3.8</em>. More besides, the transformed code also enables execution environment of <em>Static Testing Framework</em> to run test methods statically. In order to measure the degree of efficiency, the implemented tool is evaluated. The evaluation of <em>Static Testing Framework</em> draws results for different Java projects and these statistical data is compared with <em>JUnit3.8</em> results to measure the effectiveness of <em>Static Testing Framework</em>. As a result of evaluation, <em>STF</em> can be used for static creation and execution of test cases up to <em>JUnit3.8</em> where test cases are not creating within a test class and where real definition of constructors is not required. These problems can be dealt as future work by introducing a middle layer to execute test fixtures for each test method and by generating test classes as per real definition of constructors.</p>

Maintaining quality in software is a difficult task for several reasons, such as, company growth, time-to-market demands, code complexity and more. GUI testing tools and Continuous Integration (CI) are common practice today to tackle some of the issues with maintaining software quality. However, these techniques bring a set of challenges. Visual Regression Testing (VRT) is a special kind of GUI testing technique focused upon image-based assertions. This study presents an implementation and investigation of the benefits and drawbacks of introducing VRT for a CI environment in an industrial context. Additionally, the thesis investigates factors that need to be considered upon this transition. The results show that benefits are associated with this approach, such as, quicker feedback times and an increase in testing frequency. However, drawbacks and implications were also identified, such as, test maintenance and organizational concerns, indicating careful consideration needs to be taken by an organization before proceeding with an implementation.

In this modern world machine learning and deep learning have become popular choice for analysis and identifying various patterns on data in large volumes. The focus of the thesis work has been on the design of the alternative strategies using machine learning to guide the fuzzer in selecting the most promising test cases. Thesis work mainly focuses on the analysis of the data by using machine learning techniques. A detailed analysis study and work is carried out in multiple phases. First phase is targeted to convert the data into suitable format(pre-processing) so that necessary features can be extracted and fed as input to the unsupervised machine learning algorithms. Machine learning algorithms accepts the input data in form of matrices which represents the dimensionality of the extracted features. Several experiments and run time benchmarks have been conducted to choose most efficient algorithm based on execution time and results accuracy. Finally, the best choice has been implanted to get the desired result. The second phase of the work deals with applying supervised learning over clustering results. The final phase describes how an incremental learning model is built to score the test case logs and return their score in near real time which can act as feedback to guide the fuzzer.<br>I denna moderna värld har maskininlärning och djup inlärning blivit populärt val för analys och identifiering av olika mönster på data i stora volymer. Uppsatsen har fokuserat på utformningen av de alternativa strategierna med maskininlärning för att styra fuzzer i valet av de mest lovande testfallen. Examensarbete fokuserar huvudsakligen på analys av data med hjälp av maskininlärningsteknik. En detaljerad analysstudie och arbete utförs i flera faser. Första fasen är inriktad på att konvertera data till lämpligt format (förbehandling) så att nödvändiga funktioner kan extraheras och matas som inmatning till de oövervakade maskininlärningsalgoritmerna. Maskininlärningsalgoritmer accepterar ingångsdata i form av matriser som representerar dimensionen av de extraherade funktionerna. Flera experiment och körtider har genomförts för att välja den mest effektiva algoritmen baserat på exekveringstid och resultatnoggrannhet. Slutligen har det bästa valet implanterats för att få önskat resultat. Den andra fasen av arbetet handlar om att tillämpa övervakat lärande över klusterresultat. Slutfasen beskriver hur en inkrementell inlärningsmodell är uppbyggd för att få poäng i testfallsloggarna och returnera poängen i nära realtid vilket kan fungera som feedback för att styra fuzzer.

The widespread of inexpensive communication technologies, distributed data storage and web services mechanisms currently urge the collaboration among organizations. Partners are participating in this environment motivated by several advantages such as: (1) the ability to use external and professional resources, services and knowledge, (2) the reduction of time-consuming requirements and (3) the benefaction of experts experience. However, this collaboration is not perfect since several problems can arise such as the misuse of resources, disclosure of data or inadequate services. Therefore, security is an important concern of the participants. In particular trust management and access control are one of the major security issues for an organization. This thesis addresses these two areas in particular. It proposes a novel and comprehensive trust framework for Multi-Organization Environments. Our approach is organized in four parts. First, we propose a vector based model approach for defining trust vectors. These vectors evaluate a set of requirements, under conditions, and provide a degree of confidence. In our approach, we consider two different types of vectors. On the one hand, a vector that links a user to an organization and, on the other hand, a vector that links two organizations. We also show how these vectors are evaluated and shared among the different organizations, and how we combine the provided trust information in order to enhance the security. Second, the TRUST-OrBAC model was designed to add the previous trust approach to the ORBAC model. Moreover, this solution was applied with a real collaboration network between companies. Third, we present a trust ontology methodology based on access control concepts. This ontology will be used to share the trust beliefs between participants and to make equivalence between their trust objectives. How to define this trust relationship, how to understand the trust objective of a requester, and how to evaluate the recommendation value is addressed in this thesis. Fourth, we improve our work by designing a passive testing approach in order to evaluate the behavior of a user. This contribution is based on the monitoring tool MMT. Finally the entire architecture of our system is proposed

The goal of this thesis project is to improve industrial system testing efficiency. The thesis is mainly focused on the field of telecommunication. Testing is one of the most important steps before selling a product to customers and testing is also an important component of after sale maintenance. The work load for testers can be decreased and more projects could be carried out in parallel, if the testing efficiency can be improved. The thesis project is carried out within the Ericsson Product Development Unit (PDU) Packet Core System Test section. The goal is to improve the system testing efficiency of the Evolved Packet Gateway, as system testing has gradually become one of the bottle necks for the whole product release. Ericsson wants to accelerate its pace in releasing new products to the market and releasing new software to their customers. An initial investigation was made to get a clear view about how the complete system testing procedure is currently carried out and then, an analysis was made based on this information. Ideas were collected for a proposed new implementation and finally a prototype for the most useful ideas was developed and evaluated to show how the system testing efficiency can be improved. Finally, these ideas were proved to be valuable for Ericsson PDU Packet Core System Test section and two of them have been adopted and are already under developing.<br>Målet med detta examensarbete är att förbättra effektiviteten inom industriell systemtestning.  Avhandlingen är främst inriktad på området för telekommunikation. Testning är ett av de viktigaste stegen innan försäljning av en produkt till kunder och testning är också en viktig komponent under underhållsfasen. Genom att förbättra effektiviteten i testerna kan arbetsbelastningen för testare minskas och flera projekt genomföras parallellt. Examensarbetet har utförts inom Ericssons produktutvecklingsenhet (PDU) Packet Core, avdelning System Test. Avhandlingens mål är att effektivisera systemtestning av Evolved Packet Gateway (EPG), eftersom systemtest gradvis blivit flaskhalsen för hela produktens release. Ericsson vill påskynda takten som de släpper nya produkter på marknaden och ny mjukvara till sina kunder. En första undersökning gjordes för att få en tydlig bild av hur systemtest utförs för närvarande och därefter, gjordes en analys baserat på denna information.  Idéer samlades in för att föreslå en ny implementering och slutligen utvecklades en prototyp av en specifik idé och den harutvärderats för att visa hur effektiviteten hos systemtest kan förbättras. Slutligen, var dessa idéer visade sig vara värdefullt för Ericsson PDU Packet Core System Test avsnitt och två av dem har antagits och håller redan på att utveckla.

Tato diplomová práce popisuje testování REST API rozhraní aplikací. Výsledkem práce je aplikace Restty, založená na použití nástroje Swagger, která umožňuje testovat jednotlivé části API aplikací, i vytvářet a spouštět komplexní testovací scénáře nad daným rozhraním. Teoretická část práce vysvětluje principy webových služeb, vzdálených rozhraní a představuje nástroj Swagger i technologie použité k implementaci. V následující kapitole jsou v práci prezentovány designové návrhy aplikace, na které plynule navazuje kapitola o implementaci a testování, pro které je zvoleno rozhraní nástroje Red Hat JBoss BPM Suite. V závěru práce jsou vyhodnoceny přínosy aplikace Restty a navrženy případné budoucí rozšíření.

The object of this diploma thesis is the analysis of the business entity Turck, s.r.o., which operates in the field of industrial automation. Selected analytical methods evaluate the external and internal environment of the company and its financial stability. The last part of this diploma thesis provides suggestions for improvements.

Abstract Testing is an important part of quality assurance, and the use of agile processes, continuous integration and DevOps is increasing the pressure for automating all aspects of testing. Testing through graphical user interfaces (GUIs) is commonly automated by scripts that are captured or manually created with a script editor, automating the execution of test cases. A major challenge with script-based GUI test automation is the manual effort required for maintaining the scripts when the GUI changes. Model-based testing (MBT) is an approach for automating also the design of test cases. Traditionally, models for MBT are designed manually with a modelling tool, and an MBT tool is used for generating abstract test cases from the model. Then, an adapter is implemented to translate the abstract test cases into concrete test cases that can be executed on system under test (SUT). When the GUI changes, the model has to be updated and the test cases can be generated from the updated model, reducing the maintenance effort. However, designing models and implementing adapters requires effort and specialized expertise. The main research questions of this thesis are 1) how to automatically extract state-based models of software systems with GUI, and 2) how to use the extracted models to automate testing. Our focus is on using dynamic analysis through the GUI during automated exploration of the system, and we concentrate on desktop applications. Our results show that extracting state models through GUI is possible and the models can be used to generate regression test cases, but a more promising approach is to use model comparison on extracted models of consequent system versions to automatically detect changes between the versions<br>Tiivistelmä Testaaminen on tärkeä osa laadun varmistusta. Ketterät kehitysprosessit ja jatkuva integrointi lisäävät tarvetta automatisoida kaikki testauksen osa-alueet. Testaus graafisten käyttöliittymien kautta automatisoidaan yleensä skripteinä, jotka luodaan joko tallentamalla manuaalista testausta tai kirjoittamalla käyttäen skriptieditoria. Tällöin scriptit automatisoivat testitapausten suorittamista. Muutokset graafisessa käyttöliittymässä vaativat scriptien päivittämistä ja scriptien ylläpitoon kuluva työmäärä on iso ongelma. Mallipohjaisessa testauksessa automatisoidaan testien suorittamisen lisäksi myös testitapausten suunnittelu. Perinteisesti mallipohjaisessa testauksessa mallit suunnitellaan manuaalisesti käyttämällä mallinnustyökalua, ja mallista luodaan abstrakteja testitapauksia automaattisesti mallipohjaisen testauksen työkalun avulla. Sen jälkeen implementoidaan adapteri, joka muuttaa abstraktit testitapaukset konkreettisiksi, jotta ne voidaan suorittaa testattavassa järjestelmässä. Kun testattava graafinen käyttöliittymä muuttuu, vain mallia täytyy päivittää ja testitapaukset voidaan luoda automaattisesti uudelleen, vähentäen ylläpitoon käytettävää työmäärää. Mallien suunnittelu ja adapterien implementointi vaatii kuitenkin huomattavan työmäärän ja erikoisosaamista. Tämä väitöskirja tutkii 1) voidaanko tilamalleja luoda automaattisesti järjestelmistä, joissa on graafinen käyttöliittymä, ja 2) voidaanko automaattisesti luotuja tilamalleja käyttää testauksen automatisointiin. Tutkimus keskittyy työpöytäsovelluksiin ja dynaamisen analyysin käyttämiseen graafisen käyttöliittymän kautta järjestelmän automatisoidun läpikäynnin aikana. Tutkimustulokset osoittavat, että tilamallien automaattinen luominen graafisen käyttöliittymän kautta on mahdollista, ja malleja voidaan käyttää testitapausten generointiin regressiotestauksessa. Lupaavampi lähestymistapa on kuitenkin vertailla malleja, jotka on luotu järjestelmän peräkkäisistä versioista, ja havaita versioiden väliset muutokset automaattisesti

En base als reptes sorgits a conseqüència de l'escalat de la tecnologia, la present tesis desenvolupa i analitza un conjunt d'eines orientades a avaluar la sensibilitat a la propagació d'esdeveniments SET en circuits microelectrònics. S'han proposant varies mètriques de propagació de SETs considerant l'impacto dels emmascaraments lògic, elèctric i combinat lògic-elèctric. Aquestes mètriques proporcionen una via d'anàlisi per quantificar tant les regions més susceptibles a propagar SETs com les sortides més susceptibles de rebre'ls. S'ha desenvolupat un conjunt d'algorismes de cerca de camins sensibilitzables altament adaptables a múltiples aplicacions, un sistema lògic especific i diverses tècniques de simplificació de circuits. S'ha demostrat que el retard d'un camí donat depèn dels vectors de sensibilització aplicats a les portes que formen part del mateix, essent aquesta variació de retard comparable a la atribuïble a les variacions paramètriques del proces.<br>En base a los desafíos surgidos a consecuencia del escalado de la tecnología, la presente tesis desarrolla y analiza un conjunto de herramientas orientadas a evaluar la sensibilidad a la propagación de eventos SET en circuitos microelectrónicos. Se han propuesto varias métricas de propagación de SETs considerando el impacto de los enmascaramientos lógico, eléctrico y combinado lógico-eléctrico. Estas métricas proporcionan una vía de análisis para cuantificar tanto las regiones más susceptibles a propagar eventos SET como las salidas más susceptibles a recibirlos. Ha sido desarrollado un conjunto de algoritmos de búsqueda de caminos sensibilizables altamente adaptables a múltiples aplicaciones, un sistema lógico especifico y diversas técnicas de simplificación de circuitos. Se ha demostrado que el retardo de un camino dado depende de los vectores de sensibilización aplicados a las puertas que forman parte del mismo, siendo esta variación de retardo comparable a la atribuible a las variaciones paramétricas del proceso.<br>Based on the challenges arising as a result of technology scaling, this thesis develops and evaluates a complete framework for SET propagation sensitivity. The framework comprises a number of processing tools capable of handling circuits with high complexity in an efficient way. Various SET propagation metrics have been proposed considering the impact of logic, electric and combined logic-electric masking. Such metrics provide a valuable vehicle to grade either in-circuit regions being more susceptible of propagating SETs toward the circuit outputs or circuit outputs more susceptible to produce SET. A quite efficient and customizable true path finding algorithm with a specific logic system has been constructed and its efficacy demonstrated on large benchmark circuits. It has been shown that the delay of a path depends on the sensitization vectors applied to the gates within the path. In some cases, this variation is comparable to the one caused by process parameters variations.

The diploma thesis deals with the analysis of the company 1. Geo, spol. s r.o., which belongs tosmall-size companies. In the theoretical part basic concepts are clarified. The analytical part deals with the analysis of the current state of the company. Different types of analyses were realized. On the basis of the findings, several solutions leading to the improvement of the current state are proposed.

The usability group at RISE Research Institute of Sweden primarily perform usability evaluations for their clients. Usability evaluations are performed on launched products as well as early prototypes, the purpose of these evaluations is to detect problems and give suggestions on how they can be solved. The usability group offers a variety of usability evaluations, such as expert evaluations where experts assess the usability of a product and user tests were participants are observed when preforming tasks with a product. Eye tracking is a relatively new tool that measures eye movement; this tool is used within the usability group at RISE today, mostly as a compliment in usability tests. No extensive analysis is made of the data collected using eye tracking, due to lack of time, resources and knowledge. The purpose of this project is to investigate whether and, if so, how eye tracking can be used efficiently to evaluate usability in user tests. At the start of the project a literature study, an analysis of the current state was carried out along with a practical exploration of the eye tracking technology and associated analysis tools. Based on the information gathered in the first part of the project, two directions were selected. A qualitative approach, with the aim of exploring and explaining why a usability issue arises, and a quantitative approach with the aim to compare two equivalent products and determining which one of them is best in a usability perspective. Early in the project challenges with the collection and evaluating eye tracking data were identified. These challenges were largely related to how a usability test is conducted. In order to further explore how to best perform a usability test using eye tracking, a pilot study was conducted in which the test object was a microwave oven. The goal of the pilot study was to develop a test setup to be able to collect eye tracking data with minimal impact on factors affecting the eye tracking data. Examples of factors are "think aloud" and that the test moderator asks questions during when the participant perform a task. Based on the insights from the pilot study, the test setup was further developed, and both a qualitative and quantitative method of data collection and analysis was developed. The test setup and the two methods were evaluated in a usability test with two clock radios. The results of the tests in this project show that the qualitative analysis of eye tracking data can gather detailed data on how a person searches for information when performing a given task. By looking at the eye movement pattern, you can get indications of shortcomings in the information layout of a particular product. Eye tracking allows you to understand why a person fails a task by checking if the test person saw the desired information without interpreting it or if the person completely missed the information that was necessary to complete the task. The quantitative analysis of eye tracking data could not detect any differences between the products tested to indicate differences in the overall usability. The conclusion is that a qualitative analysis is the most effective way to utilize eye tracking in usability tests of consumer products.<br>Usability-gruppen på RISE Research Institute of Sweden arbetar främst med att utvärdera användbarhet på produkter för olika kunder. Användbarhets-utvärderingar utförs på lanserade produkter så väl som tidiga prototyper. Syftet med dessa utvärderingar är att upptäcka problem relaterade till användbarhet och komma med förslag på hur problemen kan lösas. Usability-gruppen erbjuder en rad olika typer av utvärderingar av produkter, exempelvis expertutvärderingar där produktens användbarhet bedöms av experter, och användartester där man observerar när en testperson använder en produkt. Eye tracking är ett relativt nytt verktyg som mäter ögonrörelser, detta verktyg används inom usability- gruppen på RISE idag som ett komplement vid använbarhets-tester. Då man saknar tid, resurser och till viss del kunskap om verktygets tillämpning används eye tracking idag endast som ett passivt verktyg. Med detta menas att man samlar in data utan att utföra några djupare analyser. Syftet med detta examensarbete är att undersöka om och i så fall hur eye tracking kan användas på ett effektivt sätt för att utvärdera användbarhet i användartester. I starten av projektet genomfördes en litteraturstudie, en nulägesanalys samt praktisk utforskning av tekniken och tillhörande analysverktyg. Baserat på den information som samlades in i första delen av projektet valdes två inriktningar. En kvalitativ inriktning, med målet att utforska och förklara varför ett visst användbarhetsproblem uppstår, och en kvantitativ inriktning med målet att jämföra två likvärdiga produkter och bestämma vilken av dessa som har bäst användbarhet. Tidigt identifierades även utmaningar med insamling och utvärdering av eye tracking-data, dessa utmaningar visade sig till stor del bero på hur ett användartest genomförs. För att vidare utforska hur man bäst genomför ett användbarhets-test med hjälp av eye tracking gjordes därför en så kallad pilotstudie där testobjektet var en mikrovågsugn. Målet med denna var att hitta ett testupplägg för att samla in eye tracking-data med minimal påverkan av omgivande faktorer som påverkar datan. Exempel på faktorer skulle kunna vara att testpersonen får ”tänka högt” eller att testledaren ställer frågor under genomförandet av en uppgift. Baserat på insikterna från pilotstudien vidareutvecklades testupplägget, och både en kvalitativ såväl som kvantitativ metod för datainsamling och analys togs fram. Testupplägget och de två metoderna utvädrades genom ett användbarhetstest där två olika klockradios testades. Resultaten från testerna som genomförts i projektet visar att man med kvalitativ analys av eye tracking data kan samla in detaljerade data på hur en person letar efter information vid utförande av en given uppgift. Genom att titta på ögonens rörelsemönster kan man få indikationer på brister i informationslayouten på en viss produkt. Med hjälp av eye tracking kan man förstå varför en person misslyckas med en uppgift genom att se om testpersonen såg den önskade informationen utan att tolka den eller om personen helt missade den information som var nödvändig för att klara uppgiften. Den kvantitativa analysen av eye tracking data kunde inte påvisa några skillnader mellan de produkter som testades för att indikera skillnader i användbarhet. Slutsatsen är att en kvalitativ analys är det effektivaste sättet att nyttja eye tracking vid användbarhetstester av konsumentprodukter.

Diplomová práce se zaměřuje na komplexní analýzu vybraného podniku NAVZAS s.r.o. Pomocí stanovených analytických metod zkoumajících interní a externí prostředí podniku je zhodnocena celková situace podniku. Současná finanční situace podniku je zhodnocena pomocí Kralickova Quick testu. Na základě provedených analýz jsou v závěru uvedeny návrhy na zlepšení stávající situace firmy na trhu.

The complexity associated with development of networked video games creates a need for tools for verifying a consistent player experience. Some networked games achieve consistency through the lockstep protocol, which requires identical execution of sent commands for players to stay synchronised. In this project a method for testing networked multiplayer lockstep games for synchronisation problems related to nondeterministic behaviour is formulated and evaluated. An integrated fuzzing AI is constructed which tries to cause desynchronisation in the tested game and generate data for analysis using log files. Scripts are used for performing semi-automated test runs and parsing the data. The results show that the test system has potential for finding synchronisation problems if the fuzzing AI is used in conjunction with the regular AI in the tested game, but not for finding the origins of said problems.<br>Komplexiteten förenad med utveckling av nätverksuppkopplade dataspel skapar ett behov av verktyg för att verifiera en konsistent spelarupplevelse. Vissa nätverksspel hålls konsistenta med hjälp av lockstep-protokollet, vilket kräver identisk exekvering av skickade kommandon för att spelarna ska hållas synkroniserade. I detta projekt formuleras och evalueras en metod för att testa om nätverksuppkopplade flerspelarspel lider av synkroniseringsproblem relaterade till ickedeterministiskt beteende. En integrerad fuzzing-AI konstrueras som försöka orsaka desynkronisering i det testade spelet och generera data för analys med hjälp av loggfiler. Skript används för att utföra halvautomatiserade testkörningar och tolka data. Resultaten visar att testsystemet har potential för att hitta synkroniseringsproblem om fuzzing-AI:n används tillsammans med den vanliga AI:n i det testade spelet, men inte för att hitta de bakomliggande orsakerna till dessa problem.

Embedded systems are everywhere! Electronic systems in just about every engineering market segment are classified as embedded systems, consumer electronics, medical, automotive, avionics, etc. Embedded systems differ from more conventional systems, such as computers, because they are limited to the embedded hardware, are designed to perform a dedicated function and have high quality and reliability requirements. Due to these characteristics, this type of system is strongly related to critical systems. Critical systems are systems that in the event of a failure can cause damage to living beings or the environment. Thus, it is necessary to ensure a high level of correctness in this type of systems. One way to increase the correctness of a system is through the process of testing. However, testing embedded systems presents a degree of difficulty because they are typically closed systems and work with real-time data that is difficult to reproduce and are non-deterministic. In this way, and with the collaboration of Altran Portugal, we intend to solve this problem by developing a framework that allows test automation for embedded systems. Automating the test data creation and execution of test case increases the quality of these systems by identifying defects to be fixed in a more efficient way. To this end, a survey of automation tools is done and each tool evaluated according to a set of criteria defined when designing the solution. The selected tool is Robot Framework, which is a widely used tool in the web and desktop application. Thus, integrating such a proficient tool in the embedded environment elevates the test automation in the embedded systems context. Then, we test the concept developed in this dissertation by executing functional tests in embedded systems that follow a model-driven development approach.

Dissertação de mestrado em Engenharia Informática<br>Primavera has invested a significant and costly man power in developing business-specific software solutions. Such solutions share a significant part of boilerplate code, namely the user interface. To minimize costs and, thus, improving it's software engineers productivity, Primavera BSS has invested many resources developing a Framework that allows for the next family of Primavera Products to be generated. The developed tool allows the Primavera Software Factory to easily adopt Software Development Processes based on Agile methodologies. The goal of this Dissertation is to add a new software component to this framework, a test automation component, that allows automated execution of tests to be performed on Products modelled on the Framework.<br>A Primavera BSS investiu muitos recursos na criação de uma Framework que permitisse gerar a próxima família de Produtos Primavera. A ferramenta desenvolvida permite à Primavera Software Factory adoptar, mais facilmente, um Processo de Desenvolvimento baseado em metodologias Agile. O objectivo desta Dissertação é o de adicionar uma nova componente a esta Framework, uma componente de Testes Automáticos, que permita executar, de forma automática, testes aos Produtos resultado da modelação efectuada na Framework e se consiga assim que as soluções de teste do Departamento de Qualidade acompanhem Processos de Desenvolvimento ágeis.

A qualidade de software tornou-se um dos mais importantes fatores diferenciadores na indústria de desenvolvimento de software. Com o objetivo de entregar produtos com mais qualidade, estão a ser adotadas estratégias para a automação de testes. No entanto, a automação tradicional necessita de conhecimentos técnicos específicos e, se não for bem planeada e implementada, pode tornar-se demasiado complexa e difícil de manter. Por isso, o desafio é utilizar/desenvolver técnicas de automação de testes que sejam eficientes, mesmo num contexto de contínua evolução dos sistemas. Este trabalho apresenta uma nova ferramenta, escalável e flexível, de automação de testes para aplicações web, baseada na metodologia keyword-driven. "Web Test Automation Framework" oferece uma plataforma de execução, implementação e gestão de testes de sistema, integração e testes de aceitação.<br>Software quality has become one of the most important differentiating factors in the software development industry. In order to deliver products with higher quality, strategies for test automation are being adopted. However, traditional test automation requires specific technical skills and if not well planned and implemented, it may end up being too complex and expensive to maintain. So the challenge is to use/develop test automation techniques that are efficient even in contexts of continuous evolving systems. This work presents a new scalable and flexible test automation framework for web based applications, based on the keyword-driven approach. Web Test Automation Framework provides a platform for the execution, implementation and management of system, integration and acceptance tests.

A qualidade de software tornou-se um dos mais importantes fatores diferenciadores na indústria de desenvolvimento de software. Com o objetivo de entregar produtos com mais qualidade, estão a ser adotadas estratégias para a automação de testes. No entanto, a automação tradicional necessita de conhecimentos técnicos específicos e, se não for bem planeada e implementada, pode tornar-se demasiado complexa e difícil de manter. Por isso, o desafio é utilizar/desenvolver técnicas de automação de testes que sejam eficientes, mesmo num contexto de contínua evolução dos sistemas. Este trabalho apresenta uma nova ferramenta, escalável e flexível, de automação de testes para aplicações web, baseada na metodologia keyword-driven. "Web Test Automation Framework" oferece uma plataforma de execução, implementação e gestão de testes de sistema, integração e testes de aceitação.<br>Software quality has become one of the most important differentiating factors in the software development industry. In order to deliver products with higher quality, strategies for test automation are being adopted. However, traditional test automation requires specific technical skills and if not well planned and implemented, it may end up being too complex and expensive to maintain. So the challenge is to use/develop test automation techniques that are efficient even in contexts of continuous evolving systems. This work presents a new scalable and flexible test automation framework for web based applications, based on the keyword-driven approach. Web Test Automation Framework provides a platform for the execution, implementation and management of system, integration and acceptance tests.

Functional Test of any given system is used to verify that the top level system is performing according to the specifications and all of the sub systems, i.e., hardware, software, inputs, outputs and sensors are operating properly. The term System is used here in context of any device or equipment consisting of hardware, software, sensors, virtual sensors and inputs / outputs. One of the examples of such a system is a semiconductor fabrication equipment. There have only been a few approaches that are used to perform the functional test of a system. Engineers typically develop custom test equipment to interface with the system under test and determine proper functioning of all the sub systems and behavior of overall system. In another method, domain experts, utilize the software of the system under test, and manually actuate / stimulate sub systems and then observe and record outcomes to determine whether the system exhibited correct behavior. A novel solution of a reusable functional test automation framework is developed. The framework interfaces with the system under test via the exposed API, and allow domain experts with minimal or no programming background to create test suites to methodically test overall functionality of the system.<br>text

碩士<br>國立高雄應用科技大學<br>資訊管理研究所碩士班<br>101<br>The Internet is becoming increasingly popular, more and more websites demand is growing stronger, with the site's functionality, the quality of the website has attracted increasing attention. Poor quality of the site is likely to cause abnormalities affect the users are still commonly used. In response to the rapidly changing business environment, the website related function of the new adjustment, the adjustment of a large number of features, it is easy to cause the system to have omissions. This study proposes a rigorous application module website automatically measured architecture, the through build test function module website, to perform the test case, so that managers can use a simple and fast method to maintain system quality, system expansion or amended can quickly find a system abnormalities. In this study, the entire site is divided into three types tested, overall site security testing, environmental testing of the system platform, functional testing of individual modules, formulated in accordance with the type of test specifications of the three standard formats, so testers through formal specification to write test cases and test cases encapsulated module. Writing test cases using the XML format, so that the test case management flexibility but also scalability. Managers in conducting functional changes or system updates, the latest version of the module will be downloaded to the site, which contains the latest code and test cases, managers can be tested directly through the website, without re-writing the test case , understanding of the system under test Log file check whether there is an exception, so that the website can be kept up to date, the best state.

Testing plays an important role in the wireless protocol stack development. In order to free the testers out of the shielded chamber, allow both the developers and the testers to use the test systems remotely and maximize the expensive test system usage. An automated test framework is highly demanded. In this thesis, the design of the automated test framework is introduced. There are four main components in the test framework. They are the front end, scheduler, test engine and data storage. The architecture and the protocol among these components are described. Further, the evaluation of the scheduler is conducted based on the queueing theory. Based on the simulation result, a good scheduling algorithm is proposed. Compared with the original scheduling algorithm, the new algorithm improves the performance of the low priority users significantly when the test systems are limited. Moreover, the detail design of the test engine is presented. With the control of the intelligent test engine, the automated test framework has the capability to launch the test cases automatically, catch the commands sent by the test system and manipulate the SUT (System Under Test) without human’s interrupt. It fulfills the objective of automation. The automated test framework has been deployed and is working well.

In this paper we introduce an analysis framework for mobile workforce automation. The framework is based on the findings from earlier research as well as on an analysis of 27 recent case studies conducted within the field of mobile workforce automation. It consists of a general reference process for mobile work and of a model explaining influencing factors (worker, task, coordination system, information system), optimization goals and their relationships in mobile business processes. The framework can be applied to process modeling, simulation, and optimization as well as to requirements analysis and return on investment calculations. Based on the results of case study evaluation, it is furthermore shown, that recent mobile IT solutions are mainly built for relatively simple processes and cooperation models. Mobilizing more complex processes still seems to be a challenge.

Tese de doutoramento em Engenharia Electrotécnica e de Computadores, na especialidade de Automação e Robótica, apresentada ao Departamento de Engenharia Electrotécnica e de Computadores da Faculdade de Ciências e Tecnologia da Universidade de Coimbra<br>Recently by progressing technology and infrastructures, monitoring and understanding human behaviour and activity is going to be more interesting in various applications. Exploring through human body parts motions to analyse human behaviours in different contexts, is the aim of this PhD research. The features of human movements are less restricted than the other possible features (e.g. facial expressions and voice) in many real world applications, however they are more complicated to be analysed. Due to the large dimensions of body parts movements and the complex dynamics and dependencies between them, high computational processing resources are needed to analyse and estimate human behaviours. Firstly, a hierarchical framework is proposed to reduce the complexity of the process in different layers. Secondly, a well-known human movement descriptor, Laban Movement Analysis (LMA), which provides different types of needed features in five components, is proposed. The LMA components prepare minimum needed features that can assist us to analyse any kind of human activities, and fill the gap between the Low Level Features (LLFs) and human movements analysis. Finally, for modeling the framework, Bayesian-based approaches (Bayesian Network (BN), Dynamic BN (DBN), and Hidden Markov Model (HMM)), are defined to deal with the uncertain data, to apply learning processes using small data, to fuse different types of features (in frequency and spatial domains) and to have enough flexibility for modeling the different dependencies between different features and layers. In this study we explored several different human activities and behaviours though the framework, namely; body parts movements, human individual analysis, human-object interaction, human-human interaction, interpersonal behaviour, and social role of people. The mentioned framework is modeled and constructed in a bottom-up strategy. During the process, different approaches are proposed to solve the problems. From the lowest level of analysis, the study is divided into two different domains; frequency and spatial. This study is performed to estimate LMA components, which provide enough knowledge in body motion level, to be able to explore more high level of human activities analysis. For instance; Effort component, which explains human body movement dynamics, is modeled in frequency domain, and Shape component, which explains human body shape deformation in 3D space during any movement, is modeled by spatial based features. Based on the mentioned components, individual human action level is modeled. In the next step to analyse human-object and human-human interaction analysis (context-based), each individual human actions and the relations between them, are needed. Those relations are modeled by inspiration of Relationship Component which is one of the less explored LMA component. Thus we were able to model human-object and human-human interactions by modeling Relationship components through the framework to explore human activities with respect to the context (i.e., scene understanding). With respect to the proposed framework, human activities in social context are explored by information of the LMA level. The system was enough flexible to provide the complex existent dependencies between different features to estimate body-motion based interpersonal behaviors and a social role, inspired by Alex Pentland investigation in "Honest signals" book. The hierarchical framework presents many capabilities such as; flexibility of modeling, generalizing to different related applications, extendability by progressing the sensory technology, dealing with uncertainly in all level of analysis, and providing semantic-based information for all layers of analysis. The proposed framework provides an automatic monitoring human behaviour system which is very highly interested application in almost everywhere that people are involved, such as; clinical study, security system, elder-care, surveillance system, sport training, virtual reality, choreography, etc. In the experimental process, a motion tracker suit which provides 3D position of human body parts in maximum 120 Hz resolution, is used. To prepare a dataset, several people dressed the suit and performed the defined activities. The attached sensors of the suit for each record trial, need to be calibrated. It means the data for each person in different trials can be different with respect to the calibration process. The obtained results in the each step, present the capability of the mentioned framework in different level of human movement activity analysis. Furthermore, a descriptive global framework to explore and estimate various level of human body-motion based activities, is proposed.<br>Os recentes progressos das tecnologias e infra-estruturas de monitorização e compreensão do comportamento humano sugerem novas e interessantes aplicações. A presente tese de doutoramento tem por objectivo a analise do comportamento humano em diferentes contextos, utilizando os movimentos do corpo humano. As primitivas associadas aos movimentos do corpo humano são menos restritivas que outras possíveis (ex. expressões faciais ou voz), no entanto a sua analise é mais complexa. O desafio prende-se com a grande dimensionalidade dos movimentos dos elementos do corpo, complexidade dinâmica, interdependência, e elevados requisitos computacionais para analise e estimação do comportamento humano. Inicialmente, propõe-se uma metodologia hierárquica para reduzir a complexidade do processo em diferentes camadas. Em seguida, propõe-se a utilização do reconhecido descritor de movimentos humanos, o Laban Movement Analysis (LMA), para extrair e decompor as primitivas em cinco componentes. As componentes LMA garantem um numero mínimo de primitivas para analise de qualquer movimento humano, colmatando a lacuna entre as Low Levels Features (LLFs) e o movimento. Finalmente, a metodologia é modelizada recorrendo a redes bayesianas (Bayesian Network (BN), Dynamic BN (DBN) e Hidden Markov Model (HMM)) permitindo lidar com a incerteza dos dados, utilizar processos de aprendizagem em pequena escala, fundir diferentes tipos de primitivas (no domínio da frequência e espaço) e obter um modelo de dependências suficientemente flexível entre as varias primitivas e camadas. A metodologia proposta nesta tese explora diferentes actividades e comportamentos humanos, nomeadamente: movimento de elementos do corpo humano, analise humana individual, interacção homem/objecto, interacção humano/humano, comportamento interpessoal e o papel social das pessoas. A metodologia proposta foi modelizada e construída seguindo uma abordagem de baixo para cima (bottom-up). A camada de analise de baixo nível foi decomposta em dois domínios: frequência e espaço. Esta camada permite estimar componentes LMA, obter um conhecimento de baixo nível dos movimentos e fornece dados para hierarquias superiores de analise de actividades humanas. Por exemplo, a componente de esforço (Effort), responsável pela dinâmica dos movimentos do corpo humano foi modelizada no domínio da frequência, a componente forma (Shape), responsável pela deformação 3D dos elementos do corpo durante os movimentos foi modelizada através de primitivas espaciais. A etapa de analise humano/objecto e interacção humano/humano (baseada no contexto) requer o conhecimento individual de cada acção humana e respectivas inter-relações. A modelização das inter-relações, foi inspirada na "Relationship component" , uma das componentes LMA pouco explorada, e permite obter uma metodologia e um modelo de interacção humano/objecto e humano/humano capaz de explorar as actividades humanas num dado contexto (scene understanding). As actividades humanas num contexto social foram igualmente analisadas utilizando informações do nível LMA. O sistema é suficientemente flexível para comportar as complexas inter-relações entre as várias primitivas utilizadas na estimação dos movimentos resultantes dos comportamentos interpessoais e respectivo papel social (inspirado no trabalho de investigação "Honest Signals" de Alex Pentaland). Uma abordagem e uma metodologia hierárquica apresentam várias vantagens, tais como modelos flexíveis, generalização a aplicações relacionadas, expansibilidade da tecnologia sensorial, capacidade de lidar com a incerteza e produção de informação semântica em todas as camadas de analise. A abordagem e o sistema proposto garantem uma monitorização automática do comportamento humano e tem uma nível de aplicabilidade muito alto em muitos cenários com pessoas, ex. estudos clinico, sistemas de segurança, cuidados a idosos, vigilância, treino desportivo, realidade virtual, coreografia, etc. O sistema proposto, foi experimentalmente testado com dados reais obtidos através de um fato com sensores de movimento capazes de gerar informação da posição 3D das várias partes do corpo humano a uma frequência de 120Hz. O dataset produzido é o resultado da utilização desse fato por varias pessoas enquanto executavam actividades bem definidas e foi calibrado individualmente. Isto significa que os dados gerados para cada pessoa nos diferentes trials podem ser diferentes consoante a calibração. Os resultados obtidos em cada etapa, demonstram as capacidades da abordagem e metodologias propostas nos vários níveis de analise da actividade humana. Em síntese, a tese propõe uma abordagem global e descritiva, suficientemente flexível, capaz de explorar e estimar os vários níveis dos movimentos do corpo baseados na actividade.

碩士<br>國立中央大學<br>資訊工程學系<br>107<br>In the past few decades, in the process of developing software products, many researchers have paid attention to software availability and software reliability, so software testing plays a very important role. Currently, the team is developing a data analysis platform for industrial production lines. The analysis platform is based on the production data of the T-factory company. The production data over the years and the experience of the professionals for many years are analyzed by the micro crack estimation software. When a certain function of the analysis platform is completed, Test to ensure the availability and correctness of this feature, and then provide it to the manufacturer to avoid future unneeded errors and improve production value. However, if the test of the user interface is tested by manpower, it takes a lot of manpower and time, and there may still be risks, and the bug cannot be detected. Therefore, it is hoped that the automated data analysis platform for this industrial production line will test automatically and implement the test cases to find out what errors and modifications need to be made on the system. Hence, the data analysis platform for industrial production lines can be used by the manufacturer quickly. This paper is to implement a test automation to test whether the function of the industrial production line data analysis platform is complete, to achieve the higher quality of the platform. In addition, I hope that our team can reduce the cost of manpower and time after I implement a test automation and focus on the development of other functions.

碩士<br>國立中央大學<br>電機工程學系<br>107<br>Testing system integration and graphical user interface play a pivotal role in the industry. This paper integrates the boomerang chart, yield, and B-score analysis, and introduces the system into the graphical user interface. This framework is more convenient for researchers to analyze. The more complete the system, the more important of platform integration and graphical user interface. In addition to making the framework more humanize, it can greatly improve the convenience of operations. Use MATLAB data storage to build a database and GUIDE to build a graphical user interface. The establishment of the database makes the data to be prepared in advance. When needed for review, the database file can be selected according to the time, and the establishment of the database can also effectively reduce the running time; The graphical user interface is designed with an appropriate file management and a well-organized database to generate the required graphics. The establishment of the platform not only reduces the difficulty of operation, but also accelerates the research effectiveness. This paper establishes a graphical analysis framework. Storing various data into a database which is used on boomerang analysis, and displaying the graphic through a graphical user interface. Researchers can avoid confusion because of the amount of data that is difficult to estimate and can experience better operating processes.

Technology scaling in the nanometer era comes with a significant amount of process variation, leading to lower yield and new types of defective parts. These challenges necessitate robust design to ensure adequate yield, and smarter testing to screen out bad chips. Statistical static timing analysis (SSTA) en- ables this but suffers from crude approximation algorithms. This dissertation first studies the underlying theories of timing graphs and proposes two fundamental techniques enhancing the core statistical timing algorithms. We first propose the refactoring technique to capture topological correlation. Static timing analysis is based on levelized breadth-first traversal, which is a fundamental graph traversal technique and has been used for static timing analysis over the past decades. We show that there are numerous alternatives to the traversal because of an algebraic property, the distributivity of addition over maximum. This new interpretation extends the degrees of freedom of static timing analysis, which is exploited to improve the accuracy of SSTA. We also propose a novel operator for computing joint probabilities in SSTA. In many SSTA applications, this is very common but is done using the max operator which results in much error due to the linear approximation. The new operator provides significantly higher accuracy at a small cost of run time. Second, based on the two fundamental studies, this dissertation devel- ops three applications. We propose a criticality computation method that is essential to robust design and test synthesis; The proposed method, combined with the two fundamental techniques, achieves drastic accuracy improvement over the state-of-the-art method, demonstrating the benefits in practical ap- plications. We formulate the statistical path selection problem for at-speed test as a gambling problem and present an elegant solution based on the Kelly criterion. To circumvent the coverage loss issue in statistical path selection, we propose a testability driven approach, making it a practical solution for coping with parametric defects.<br>text

Testing every possible combination of the input parameter values is often impractical, inefficient or too expensive. One common alternative is pairwise testing where every pairwise combination of the parameter values is tested. Although pairwise testing significantly reduces the number of test cases, the challenge remains in analyzing the test outputs to discern the precise characteristics of parameters causing the failures. This thesis proposes a novel approach to output analysis by identifying “bad pairs”: pairs that always result in failed test cases. A framework implementing the proposed approach is presented together with three case studies. Results from the case studies suggest there are positive relationships among the numbers of failed test cases, faults, and independent bad pairs. Also, filtering of test cases seems to have a significant impact on the bad pairs identified. We believe the proposed approach can facilitate the debugging process in software testing.

碩士<br>國立高雄師範大學<br>華語文教學研究所<br>97<br>This thesis presents an empirical study which analyzes the correspondence between test results of language proficiency and the communication skills of language learners in daily life. We design a set of oral tasks based on the Common European Framework (CEF) and the communication intension of testee. These tasks can be divided two types, namely monolog and interaction, which are used to comprehensively grade the oral ability of 30 Chinese learners who has attended the Test of Proficiency- Huayu (TOP). We first analyze the distribution of the graded oral ability with respect to CEF for three types of learners, namely those who fail the basic level of TOP (Type-I), those who pass the basic level of TOP but fail the intermediate level (Type-II) and those who pass the intermediate level of TOP (Type-III). Then we examine the correspondence between the testee’s oral ability and their TOP grades in listening, grammar as well as reading. The analysis results show that 1) the testee outperforms in interactive oral tasks than in monolog ones, 2) the oral grade of the testee with respect to CEF is higher than the CEF one transformed from their TOP grades, 3) the oral ability of the testee who take the basic test is relevant to their grade in listening and grammar, 4) the oral ability of the testee who take the intermediate test is relevant to their overall grade in TOP and 5) there exits intermediate correspondence between the testee’s grades in TOP and their oral grades in CEF. These results clarify the relationship between the oral ability and the language proficiency tests, showing that under interactive conditions, testee has higher oral grades than those quantified by their TOP test results. These results indicate that the development of Chinese oral tests should take the communication intension into account. Also the spoken Chinese teaching class should especially emphasize grammar and communication skills of students. This study analyzes the relationship between CEF and the learners’ oral performance, which can be good references in developing oral tests for the Chinese proficiency test community. Furthermore this study is the first one that identifies a correspondence of TOP by designing oral tasks based on CEF. In this manner, we expect that CEF would receive more and more attention in the research for applications of Chinese.