TECHNICAL FIELD

[0001] This application relates to performance management for industrial systems.

BACKGROUND

[0002] The current industry-standard Wind turbine performance monitoring technology is based on a mapping of wind speed versus extracted energy. This mapping is called a power curve. The International Electrotechnical Commission standardized the estimation of the power curve using a nonparametric approach, known as the“binning method”. Given the wind speed and extracted energy observation data, the binning method divides the domain of wind speed into a finite number of intervals or bins. The power curve is then obtained as a mapping of sample mean of wind speed versus sample mean of extracted energy using all observation data falling within each bin. For a given wind speed, the corresponding extracted energy is the sample conditional expected value of the extracted energy over the period considered:

E[K5 ?] .

[0003] Once constructed, the power curve is then used to predict extracted energy at any wind speed, by mapping or interpolation. This methodology requires prior construction of power curves, which may be meaningful only after observing data over a sufficiently long period of time.

[0004] More recent methodology proposes to use machine learning to create multi- variable models of turbine performance, when sufficient information is available. An ensemble of regression trees to predict turbine extracted energy as a function of several inputs, including shear, wind speed, and hub height turbulence has been proposed. These methods are instances of the“random forest” method. Other approaches include a list of commonly-used input variables used in an additive multivariate kernel method that can include the aforementioned weather factors as a new power curve model. Note that a power curve, in this case, is a high-dimensional manifold. Multi-variable methods require prior construction of high-dimensional power curves, which may be meaningful only after observing data over a sufficiently long period of time. In essence, they require more data and are more difficult to understand and to implement than the current standard practice.

SUMMARY

According to aspects of embodiments of a method for detecting performance loss in a cooperative system comprising a plurality of agents, a performance analyzing processor receives production information from the plurality of agents and calculates a performance index for each agent in the plurality of agents. A performance graph having each state of an agent as a vertex and edges connecting two vertices if a first vertex is strongly preferred to a second vertex is constructed and a performance score matrix is constructed in part on the performance graph. A ranking of agents is produced in decreasing order of performance by ordering the components of the performance score matrix. The ranking may be included in reporting information provided to the users. In some embodiments, the plurality of agents is a plurality of wind turbines and the cooperative system is a wind farm. In those embodiments, the production information includes state information of each wind turbine and control information for each wind turbine. The performance score matrix may be constructed as an n x n matrix as a convex combination of n x n link matrices. A ranking score vector of the performance score matrix S is defined as a solution to the linear equation x = Sx. According to other embodiments, updated control information may be provided to one or more of the agents in the plurality of agents based on a corresponding performance index. In some embodiments, the method may further include estimating an extractable resource for each state of one of the agents, comparing an extracted resource from the one of the agents to the estimated extractable resource for a given state and calculating the performance index based on the comparison.

According to aspects of embodiments of systems described in this disclosure, a system for detecting performance loss in a cooperative system comprising a plurality of agents includes a performance analyzer configured to receive production information from the plurality of agents and calculate a performance index for each agent in the plurality of agents, construct a performance graph having each state of an agent as a vertex and edges connecting two vertices if a first vertex is strongly preferred to a second vertex and based at least in part on the performance graph create a performance score matrix, based in part on the performance graph to rank the plurality of agents according to performance. The plurality of agents is a plurality of wind turbines and the cooperative system is a wind farm. The production information includes state information of each wind turbine and control information for each wind turbine. In some embodiments the performance analyzer may be further configured to construct the performance score matrix as an n x n matrix as a convex combination of n x n link matrices. A ranking score vector may be extracted from the performance score matrix S is a solution to the linear equation x = Sx. According to other embodiments, an interface between the performance analyzer and the plurality of agents providing updated control information to one or more of the agents in the plurality of agents based on a corresponding performance index. For some embodiments, the performance analyzer may calculate the performance index by estimating an extractable resource for each state of one of the agents, comparing an extracted resource from the one of the agents to the estimated extractable resource for a given state and calculating the performance index based on the comparison.

BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The foregoing and other aspects of the present invention are best understood from the following detailed description when read in connection with the accompanying drawings. For the purpose of illustrating the invention, there is shown in the drawings embodiments that are presently preferred, it being understood, however, that the invention is not limited to the specific instrumentalities disclosed. Included in the drawings are the following Figures:

[0006] FIG. 1 shows a multi-agent wind turbine installation according to embodiments of the present disclosure.

[0007] FIG. 2 is a block diagram of a system for monitoring performance of turbines in a multi-agent system according to aspects of embodiments of this disclosure.

[0008] FIG. 3 is a block diagram of a method for monitoring performance of turbines in a multi-agent system according to aspects of embodiments of this disclosure.

[0009] FIG. 4 is a block diagram DETAILED DESCRIPTION

[0010] A wind turbine is a controlled system that converts kinetic energy from the ambient weather into electric energy. Wind turbines are an important asset in the renewable energy production portfolio. The U.S. Department of Energy (DOE) supports research and development towards achieving a 20% penetration of wind in electricity production in the United States by 2030. Wind turbines generally operate in remote sites under random weather conditions. It is often costly to access these remote locations. A site may be onshore or offshore.

[0011] FIG. 1 depicts a sample multiple-turbine offshore site 100. An array of wind turbines is arranged to convert wind energy to electrical energy. A first wind turbine 101 operates to other wind turbines, for example turbine 103 and turbine 105. It is of paramount energy relevance to monitor fleets of turbines in order to detect any under- performing turbines, to improve their energy extraction efficiency, or to schedule maintenance. Much of the research on wind turbine performance optimization has focused on single turbine settings. Maximizing the performance of turbines in a multi- turbine site is more complex due to inter-turbine aerodynamics. Hence much of the existing technology is highly inefficient for optimizing energy extraction from wind turbines.

[0012] For various reasons including environmental, social or political reasons, wind turbines are often located in remote locations. For example, wind turbine farms may be located at offshore locations. In these remote locations, it is difficult to detect or experience the conditions in which the turbine is operating. Dynamic environmental conditions create non-stationary or unknown probability terms that may affect the efficiency or operation of the turbines. The basic operation of a wind turbine is to convert the kinetic energy of air in motion as wind, into electrical energy through the rotation of a blade placed in the path of the airflow. Therefore, the main input variable for analyzing the performance of the turbine is wind speed and the main output variable is the electrical energy generated. However, the observed electrical output does not provide information as to how well the turbine is performing and the question exists as to whether given the current conditions, could the turbine perform more efficiently and generate more electrical energy than is being observed.

[0013] FIG. 2 provides a block diagram of a system for monitoring performance of agents in a multi-agent cooperative system. A wind farm 100 is subject to environmental factors 210, including wind speed and direction that allows the conversion of kinetic energy of air represented as wind speed, to electrical energy derived from force applied to the surface of the turbine blades. Information relating to the conversion of wind to electrical energy creates production data 220. The conversion data is presented to a performance analyzer 201. The performance analyzer 201 may include a computer processor specifically configured for receiving, processing and acting on conversion data from a multi-agent system, such as the multi-agent system wind farm 100. The performance analyzer 201 performs analysis of the production data 220 and may calculate a performance index for each turbine in wind farm 100. A classifier 221 is in communication with the performance analyzer and identifies subsets of turbines (e.g., agents) that have identical configurations. Agents having identical configurations may be expected to convert the same amount of energy, all factors being equal. These performance comparable agents may be compared to identify specific agents that are not performing as expected. Further processing of the production data 220 is used to construct a performance score matrix that ranks the turbines in decreasing order of performance 203. Based on the ranking, comparative performance reporting 205 may be presented to a user. Based on the ranking, under-performing agents may be identified and corrective action may be taken. Corrective action may include, but is not limited to, updating control information of the turbine or scheduling maintenance on an under-performing agent. In some embodiments, an optimization may be performed to generate turbine control data 230 that is determined to improve the performance of the identified wind turbine. The turbine control data 230 is fed back to the turbine in the wind farm 100 to effect the controls of the wind turbine to improve the wind turbine’s performance. The wind turbine will provide updated production data 220 which will again be analyzed by performance analyzer 201 to determine the updated operation state of the turbine.

[0014] Presented herein below is a method for estimating the efficiency loss of a class of multi-agent stochastic dynamic systems, resource allocation or conversion systems, being application examples. We require no construction of any actual or statistical performance curves. The method is data-driven, requiring the observation of only sensitive state and action variables for each agent. An estimator of extractable resource is generated, and a new performance index for each agent is computed. A ranking mechanism is presented based on these performance indices. We use the indices and ranking to formulate the performance maximization problem as a stochastic dynamic game. Preliminary numerical results showed that the proposed method successfully captured some real cases of under-performance. [0015] Notation used throughout this description can be summarized as follows. A lowercase letter (e.g., x) denotes a vector. An uppercase letter (e.g., A) denotes a matrix. A calligraphic symbol denotes a feasible set. For example, the symbol X denotes the feasible set of the variable x. Latin lowercase letters are used to denote primal variables. Lowercase Greek letters are used to denote dual variables. If x is a variable, then x and x denote its lower bound and upper bound, respectively. If i is an element of a countable set S, we write -i A £{i}; that is all the elements of the set, except /. If x denotes a vector, then n _x ; and X denote the dimension and feasible set, respectively, of x. E[x] or Ex is used as shorthand for the expected value of x. We write E[x|y] as the conditional expectation of x given y. Let {y _t } be a sequence of random vectors. The symbol H denotes the Hilbert space spanned by the vectors {yo, - - - ,y _t }; that is, H =span{y ₀ ,..., y _t+ }. The symbol denotes the Hilbert space spanned by y _t (i.e. at time t only): = span {y _t }. We write y _t \ _s to denote the projection onto H .

If V is a vector space, dim(V) denotes the dimension of V. Other symbols with a narrower scope, such as auxiliary variables, in the subsections where they are used. The symbol denotes a binary relation on some set of alternatives X, allowing the comparison of pairs of alternatives x, y e X. The notation x > y is shorthand for,“x is at least as good as y”. The binary relation symbols > and ~ denote string or strong preference and indifference, respectively.

[0016] The problem is to monitor the performance of a class of dynamic system of distributed cooperative agents. Resource allocation and conversion systems are examples of application. There are multiple agents. The agents operate in unknown environmental conditions. The agents are coupled (or networked) via the environment or their actions.

• A tool is required, which shall monitor, automatically or on-demand, the subset of underperforming agents.

• The tool shall compute an estimate of the performance degradation.

• The tool shall submit its findings to a remote diagnostics team for further evaluation and actions.

[0017] A wind turbine is a controlled system that converts kinetic energy from the ambient weather into electric energy. Wind turbines are an important asset in the renewable energy production portfolio. The DOE supports research and development towards achieving a 20% penetration of wind in electricity production in the United States by 2030. Wind turbines generally operates in remote sites, under random weather conditions, and often costly-to-access sites. A site may be onshore or offshore. It is of paramount energy relevance to monitor fleets of turbines in order to detect any under-performing turbines, to improve their energy extraction efficiency, or to schedule maintenance. Much of the research on wind turbine performance optimization has focused on single turbine settings. Maximizing the performance of turbines in a multi- turbine site is more complex due to inter-turbine aerodynamics. Hence much of the existing technology is highly inefficient for optimizing energy extraction from wind turbines. The current industry standard wind turbine performance monitoring technology is based on a mapping of wind speed versus extracted energy. This mapping is called a power curve. The International Electrotechnical Commission standardized the estimation of the power curve using a nonparametric approach, known as the“binning method”. Given the wind speed and extracted energy observation data, the binning method divides the domain of wind speed into a finite number of intervals or bins. The power curve is then obtained as a mapping of sample mean of wind speed versus sample mean of extracted energy, using all observation data falling within each bin. For a given wind speed, the corresponding extracted energy is the sample conditional expected value of the extracted energy over the period considered: E\y \x* ^b ]. Once constructed, the power curve is then used to predict extracted energy at any wind speed, by mapping or interpolation. This methodology requires prior construction of power curves, which may be meaningful only after observing data over a sufficiently long period of time. More recent methodology proposes to use machine learning to create multi-variable models of turbine performance, when sufficient information is available. An ensemble of regression trees has been suggested to predict turbine extracted energy as a function of several inputs, including shear, wind speed, and hub height turbulence. These methods are instances of the“random forest” method. Other research presents a list of commonly-used input variables. Also proposed is an additive multivariate kernel method that can include the aforementioned weather factors as a new power curve model. Note that a power curve, in this case, is a high-dimensional manifold. Multi-variable methods require prior construction of high-dimensional power curves, which may be meaningful only after observing data over a sufficiently long period of time. In essence, they require more data and are more difficult to understand and to implement than the current standard practice.

[0018] Consider a set J c M of performance-comparable wind turbines commissioned in a geographic site. Let x - , for all i e 3, denote the multi-variable state process with components. Let u\ ^b for all i e J , denote the multi-variable control process with components Ug ^b _sit , u ^tb _atbk , and u ^tb _vit . Let u ^tb _pit for all i e J denote the extracted resource.

Assume that the weather variables can be represented by stochastic differential equations driven by the Wiener processes. For simplicity, without loss of generality, consider Ito drift-diffusion processes, for t e T: dx _it — u _it dt + a _it dw _it , Vi 6 J, (1 ) where wi _t , for all / are standard Wiener processes. The first and second terms on the right-hand side of the equation are drift and diffusion components, respectively.

[0019] For notational conciseness and simplicity, the differential forms in Equation (1 ) are used interchangeably with their respective integral forms or discrete approximations. Accordingly, Equation (1 ) is equivalent to:

where the last term is an Ito integral.

[0020] The use of stochastic differential equations is common in various fields dealing with weather factors. It has been proposed that a drift-diffusion process be used to model temperature dynamics with seasonal stochastic volatility. The model is used to compute future prices for commonly traded contracts in the Chicago Mercantile Exchange on indices like cooling and heating degree days, and cumulative average temperatures, as well as option prices on these indices. Synoptic variability of mid- latitude seas surface winds (obtained from 6 hourly scatter ohmmeter observations) can be well described using a univariate ltd drift-diffusion stochastic differential equation. [0021] Some definitions used throughout this description will now be described.

[0022] An extractable resource d _it is a function of the state variables, independent of the agents’ actions: d _it A d _it (x _it ).

[0023] An extracted resource is a function of the state and agents’ action variables: tb tb

y apit— y apit

[0024] P-Comparable Turbines are turbines that for two turbines / and j, for any i,je ΰ are performance comparable, or p-comparable if thy have identical design characteristics. Note that any turbine is p-comparable with itself. Given i,j, fe e J, if / is p- comparable with j, then j is p-comparable with /; if / is p-comparable with j and j is p- comparable with k, then / is p-comparable with k. Hence the collection of p-comparable turbines defines a partition of J.

[0025] Strongly P-Comparable States - Let i,j e J be a pair of p-comparable turbines. Let, s, t e [0, 7] be two time instants. Let x . ^b _is and xi ^b _;-t be the corresponding weather process state sub-vectors; i.e. , excluding the wind speed components. Then x- ^b and xft are strongly p-comparable, if

[0026] Note that any state is strongly p-comparable with itself. Given three sub-vectors f xl ^b _x is strongly p-comparable with xl ^b ₂ , then xl ^b ₂ is strongly p- If xl ^b _x is strongly p-comparable with xl ^b ₂ and xl ^b ₂ is strongly p- comparable with xl ^b ₃ , then xl ^b _sl is strongly p-comparable with xl ^b ₃ . Hence the collection of strongly p-comparable states defines a partition of the set of states. While the concept of strongly p-comparable states is easily understood, the referenced conditions may be somewhat restrictive for practical purposes. For example, as stated above, the wind industry currently stores data averaged over 10-minute intervals. Moreover, some physical variables (e.g. turbulence) may be well-defined over a non-vanishing time interval. Therefore, a concept of a weakly P-Comparable State is useful and defined below.

[0027] Weakly P-Comparable States- Let i,j e ? be a pair of p-comparable turbines. Let s, t e [0, 7] be two time instants. Let xi ^b _sis and xi ^b _sjt be the corresponding weather process state sub-vectors; i.e. excluding the wind speed components. Then xf and xfj* are p-comparable, if there exists At > 0 such that:

[0028] Equation (5) is equivalent to the following conditions:

[0029] Extractable Power Estimator - Let s _it be the set of all states that are non- anticipatively p-comparable with the state of turbine i e J at time t e [0, 7] Then a (non- anticipative) estimator of the extractable resource of / at t is given by:

[0030] Based on this definition, a novel performance index is produced for each state as a measure of under-performance of a turbine. Recalling that a state is defined by a turbine and a time t e [0, 7] the performance index is defined as the new index as the extracted resource below an adapted threshold. Note that, by definition, >

E / yi _jT d . Hence, y f _it > 0. Integrating over the period [0, t] yields a corresponding performance energy index: yiiit = Jo y¹iT άt, nΐ E 0, Vt E T (9)

[0031] FIG. 3 is a block diagram of a method for ranking a plurality of agents of a multi agent system, like the system of wind turbines presented in FIG. 1 and FIG. 2. The wind farm generates data 301 including state variables relating to each wind turbine in the wind turbine system. The state variables are received by a performance analyzer 303. In addition, the control variables relating to the wind turbine are received by the performance analyzer 305. A performance index for each turbine is calculated as the difference between the extracted resource as compared to a threshold that represents full performance of the wind turbine 307. The performance index of a first turbine is used to create a performance score matrix 309. The performance score matrix ranks the turbines in order of performance identifies wind turbines that are underperforming based on the performance indexes the ranking information produces reporting information of under-performing turbines 31 1. The user may take corrective action based on the reporting.

[0032] The mapping -y^ ( ) is a utility function on p-comparable equivalence classes. Intuitively, if s ^tb / and s ^tb / belong to the equivalence class. Then, the following is true:

[0033] Positive Matrix - A matrix S is positive if S a > 0 for all / and j.

[0034] Column-Stochastic Matrix - An n ^c n matrix L is a column-stochastic matrix if all its components are nonnegative and the sum of all components in each column is equal to one; that is = 1 , for all j, respectively.

[0035] Performance Graph- Consider n p-comparable states, n > 1 , (x^, k = 1, ... , n ). A performance graph is a directed graph G(V, E) Where V and E denotes the sets of vertices and edges, respectively. Each state represents a vertex. A directed edge is defined from some vertex e, to some vertex e ₇ if and only if the state s, is strongly preferred to s _j .

[0036] Given the above definitions, there are two problems faced. The first is estimation and the second is performance maximization. For the estimation problem, given observations on the weather, control and extracted resource processes, z (. x, u, z ) over a time period [0 ,t], find a function h° in the class of measurable functions that satisfies:

E _Y tb

^A ap it(t+x) ft°({ _z ' ^b }') < 00 (1 1 )

[0037] Regarding performance maximization, the problem is to find feasible control processes u^, and u^ _pi , for all i e 3, that extracts the closes value from the extractable resource, or more formally:

[0038] Clearly, barring any binding operational or design constraints, each turbine operates insomuch as to efficiently convert the available kinetic energy into electric energy. This is achieved by automatically adjusting the control variables. In basic terms, it may be said that any turbine whose operation deviates from a given maximum energy extraction engineering design principle is under-performing. Performance monitoring aims to detect any under-performing turbine. Performance maximization aims at improving the performance of any under-performing turbine.

[0039] A new method for solving the estimation and performance maximization problems is described with the following assumptions. First, the extracted energy is a monotone non-decreasing function of wind speed, for a fixed air density. More formally, given a turbine, say i e J, and two time instants, say t _it t ₂ e T, if every variable or parameter, except the wind speed that is correlated with the extracted energy variable remain constant in probability, then the following condition must hold in probability:

[0040] Thus, any violation of the necessary condition in equation (13) is an indication of under-performance of turbine / at time t ₂ .

[0041] Further, given two turbines, say i,j e J, and a time instant, say e T, if every variable or parameter, except the wind speed, that is correlated with the extracted energy variable, are identical in probability, then the following condition must hold in probability:

[0042] Thus, any violation of the necessary condition in equation (14) is an indication of under-performance of turbine j at time .

[0043] Presently, industry standard wind turbine performance monitoring practice recommends that data be acquired, averaged over 10 minutes, and stored, because wind speeds are considered stationary and other environmental factors nearly constant over a 10-minute period. The systems and methods of the present invention are improvements that eliminate the need for creating complex power curves that require extended periods of operations monitoring to be useful.

[0044] A novel performance ranking score is now introduced for ranking wind turbines over a given time period based on their performance. The performance ranking score uses efficiency loss estimates using the concept of positive column stochastic matrices and related linear algebra techniques to produce a meaningful performance ranking score. Based on the definitions for positive and column-stochastic matrices it will be understood that every column stochastic matrix has an eigenvalue of one. If a matrix S is positive and column stochastic, any eigenvector in S has all positive or all negative components. If v and w are linearly independent vectors in in M ⁿ , n > 2. Then, for some values of s and t that are not both zero, the vector x = sv + tw has both positive and negative components. Finally, if S is positive and column-stochastic then = 1. [0045] With the aforementioned principles, a positive column stochastic matrix S based on the performance index concept is presented. Given S, solve the linear algebraic system Sx=x subject llxHi = 1 to obtain the unique solution x ^* . Ordering the components of x ^* in decreasing order yields a unique ranking of turbines in decreasing order of performance.

[0046] The performance score matrix may be constructed as an n x n matrix S as a convex combination of n x n link matrices _; , i e {0,1,2} according to: a ₀ e (0,1], a , a ₂ E [0.1], (15a) a ₀ + a _± + a ₂ = 1, (15b)

[0047] Where Ui _j is a binary variable that is equal to1 if there is a link from node / to j, or 0, otherwise; p _/7 denotes the performance index of / with respect to Here, we use the convention - := 0.

0

[0048] The ranking score vector is the solution to the following linear equation: x = S _x . (16) [0049] FIG. 4 illustrates an exemplary computing environment 400 within which embodiments of the invention may be implemented. Computers and computing environments, such as computer system 410 and computing environment 400, are known to those of skill in the art and thus are described briefly here.

[0050] As shown in FIG. 4, the computer system 410 may include a communication mechanism such as a system bus 421 or other communication mechanism for communicating information within the computer system 410. The computer system 410 further includes one or more processors 420 coupled with the system bus 421 for processing the information.

[0051] The processors 420 may include one or more central processing units (CPUs), graphical processing units (GPUs), or any other processor known in the art. More generally, a processor as used herein is a device for executing machine-readable instructions stored on a computer readable medium, for performing tasks and may comprise any one or combination of, hardware and firmware. A processor may also comprise memory storing machine-readable instructions executable for performing tasks. A processor acts upon information by manipulating, analyzing, modifying, converting or transmitting information for use by an executable procedure or an information device, and/or by routing the information to an output device. A processor may use or comprise the capabilities of a computer, controller or microprocessor, for example, and be conditioned using executable instructions to perform special purpose functions not performed by a general purpose computer. A processor may be coupled (electrically and/or as comprising executable components) with any other processor enabling interaction and/or communication there-between. A user interface processor or generator is a known element comprising electronic circuitry or software or a combination of both for generating display images or portions thereof. A user interface comprises one or more display images enabling user interaction with a processor or other device.

[0052] Continuing with reference to FIG. 4, the computer system 410 also includes a system memory 430 coupled to the system bus 421 for storing information and instructions to be executed by processors 420. The system memory 430 may include computer readable storage media in the form of volatile and/or nonvolatile memory, such as read only memory (ROM) 431 and/or random access memory (RAM) 432. The RAM 432 may include other dynamic storage device(s) (e.g., dynamic RAM, static RAM, and synchronous DRAM). The ROM 431 may include other static storage device(s) (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, the system memory 430 may be used for storing temporary variables or other intermediate information during the execution of instructions by the processors 420. A basic input/output system 433 (BIOS) containing the basic routines that help to transfer information between elements within computer system 410, such as during start-up, may be stored in the ROM 431. RAM 432 may contain data and/or program modules that are immediately accessible to and/or presently being operated on by the processors 420. System memory 430 may additionally include, for example, operating system 434, application programs 435, other program modules 436 and program data 437.

[0053] The computer system 410 also includes a disk controller 440 coupled to the system bus 421 to control one or more storage devices for storing information and instructions, such as a magnetic hard disk 441 and a removable media drive 442 (e.g., floppy disk drive, compact disc drive, tape drive, and/or solid state drive). Storage devices may be added to the computer system 410 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire).

[0054] The computer system 410 may also include a display controller 465 coupled to the system bus 421 to control a display or monitor 466, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. The computer system includes an input interface 460 and one or more input devices, such as a keyboard 462 and a pointing device 461 , for interacting with a computer user and providing information to the processors 420. The pointing device 461 , for example, may be a mouse, a light pen, a trackball, or a pointing stick for communicating direction information and command selections to the processors 420 and for controlling cursor movement on the display 466. The display 466 may provide a touch screen interface which allows input to supplement or replace the communication of direction information and command selections by the pointing device 461. In some embodiments, an augmented reality device 467 that is wearable by a user, may provide input/output functionality allowing a user to interact with both a physical and virtual world. The augmented reality device 467 is in communication with the display controller 465 and the user input interface 460 allowing a user to interact with virtual items generated in the augmented reality device 467 by the display controller 465. The user may also provide gestures that are detected by the augmented reality device 467 and transmitted to the user input interface 460 as input signals. [0055] The computer system 410 may perform a portion or all of the processing steps of embodiments of the invention in response to the processors 420 executing one or more sequences of one or more instructions contained in a memory, such as the system memory 430. Such instructions may be read into the system memory 430 from another computer readable medium, such as a magnetic hard disk 441 or a removable media drive 442. The magnetic hard disk 441 may contain one or more datastores and data files used by embodiments of the present invention. Datastore contents and data files may be encrypted to improve security. The processors 420 may also be employed in a multi-processing arrangement to execute the one or more sequences of instructions contained in system memory 430. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.

[0056] As stated above, the computer system 410 may include at least one computer readable medium or memory for holding instructions programmed according to embodiments of the invention and for containing data structures, tables, records, or other data described herein. The term“computer readable medium” as used herein refers to any medium that participates in providing instructions to the processors 420 for execution. A computer readable medium may take many forms including, but not limited to, non-transitory, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such as magnetic hard disk 441 or removable media drive 442. Non-limiting examples of volatile media include dynamic memory, such as system memory 430. Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the system bus 421. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0057] The computing environment 400 may further include the computer system 410 operating in a networked environment using logical connections to one or more remote computers, such as remote computing device 480. Remote computing device 480 may be a personal computer (laptop or desktop), a mobile device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer system 410. When used in a networking environment, computer system 410 may include modem 472 for establishing communications over a network 471 , such as the Internet. Modem 472 may be connected to system bus 421 via user network interface 470, or via another appropriate mechanism.

[0058] Network 471 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication between computer system 410 and other computers (e.g., remote computing device 480). The network 471 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-6, or any other wired connection generally known in the art. Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generally known in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in the network 471.

[0059] An executable application, as used herein, comprises code or machine- readable instructions for conditioning the processor to implement predetermined functions, such as those of an operating system, a context data acquisition system or other information processing system, for example, in response to user command or input. An executable procedure is a segment of code or machine-readable instruction, sub-routine, or other distinct section of code or portion of an executable application for performing one or more particular processes. These processes may include receiving input data and/or parameters, performing operations on received input data and/or performing functions in response to received input parameters, and providing resulting output data and/or parameters.

[0060] A graphical user interface (GUI), as used herein, comprises one or more display images, generated by a display processor and enabling user interaction with a processor or other device and associated data acquisition and processing functions. The GUI also includes an executable procedure or executable application. The executable procedure or executable application conditions the display processor to generate signals representing the GUI display images. These signals are supplied to a display device which displays the image for viewing by the user. The processor, under control of an executable procedure or executable application, manipulates the GUI display images in response to signals received from the input devices. In this way, the user may interact with the display image using the input devices, enabling user interaction with the processor or other device. [0061] The functions and process steps herein may be performed automatically or wholly or partially in response to user command. An activity (including a step) performed automatically is performed in response to one or more executable instructions or device operation without user direct initiation of the activity.

[0062] The system and processes of the figures are not exclusive. Other systems, processes and menus may be derived in accordance with the principles of the invention to accomplish the same objectives. Although this invention has been described with reference to particular embodiments, it is to be understood that the embodiments and variations shown and described herein are for illustration purposes only. Modifications to the current design may be implemented by those skilled in the art, without departing from the scope of the invention. As described herein, the various systems, subsystems, agents, managers and processes can be implemented using hardware components, software components, and/or combinations thereof.