External Software

Definitions

Software Categories

• Where Is Needed: It shows how neccessary is to have the package on every machine or every place. It covers the accesibility of the software.
• Everywhere: Should be available on each machine running Atlas software. (Example: Control).
• Nearby: Should be available to everyone, but not necessarily on every machine. Everyone should have access to machine capable of running them. (Example: Calibration).
• Somewhere: May be missing. Will add additional functionality, which is in general not required but may be useful for specific tasks. (Example: Presentation Graphics).
• Replaceability: It shows how much is the rest of the Atlas software dependent on the package and how easy or difficult is to replace it.
• Core: Is in the core system. Most of other packages heavily depend on it. It is not simple to replace it by other package. It influences heavily design of the rest of the Atlas software. (Example: Persistent Storage).
• Replaceable: Can be replaced by other package without big problems. Have  little or no impact on the design of the rest. (Example: Mathematics).
• One-of-many: Several packages with similar functionality exist possibly even at the same time. User may use any of them in a similar way. (Example: GUI).
• Impact: It shows the impact on the data and results. Some packages may for example make software more user-friendly, but would not change neither the stored data nor the physical results.
• Data: Have direct impact on the stored (persistent) physical data. (Example: Reconstruction).
• Results: Have only impact on analyses results. (Example: Analysis).
• No: Have no direct impact on data or analyses results. The graphical presentation may, for example, change, but numbers shouldn't. Possible error would not infect directly the physical results. (Example: GUI).

Package Evaluation

1. Anyone can ask for evaluation of software package. She also supplies the filled questionaire together with additional information which may be usefull for evaluation and possibly suggestion for evaluators (if special technical items should be addressed). The application goes directly to concerned Domain. Domain tries to get evaluation applications for similar (competing) products at the same time, so it can make reasonable comparison.
2. Domain decides if:
1. The package belongs to that Domain.
2. The package can be evaluated as external package. Note, that only packages which have been reviewed, rump-uped or evaluated are allowed to be used in Atlas for obtaining results for eventual publication.
3. Which category package belongs to (approximately).
Domain then sends the request to DIG. It may add any relevant additional information and suggest evaluators.
3. DIG chooses approximately 3 to 5 evaluators and sends them all available documentation and questionnaire. It also suggests them the category of the package. The evaluators will be mostly internal from Atlas as satisfying Atlas constraints is important requirement. Some evaluators may be from HEP outside of Atlas or even from outside of HEP (to be able to judge more technical items).
4. Evaluators perform evaluation of the package (including comparison with similar packages which are available) and send the filled questionnaire with any relevant comments back to Domain. This period may require evaluation and testing.
5. Domain suggests accepting or denying of the package and planed schema of package handling to DIG. Domain may perform another checks or testing.
6. DIG decides about package acceptance and schema of package handling. If any finacial decision is required for the package acceptance,  the decision of DIG should be confirmed by ACOS.

1. To which categories does it belong ? Make your own judgement, even if categories are already suggested.
2. Is it useful ? Does it satisfy Atlas requirements ? Does it satisfy any particular requirement (not yet satisfied or badly satisfied) ?
1. In which timescale is it required ? Does it satisfy immediate need or more long-term strategy ? Is it needed now, in medium term, in long term ?
3. What is the price ? The price for the whole Atlas collaboration is important. Include also HEP manpower.
1. What is the maintenance and support price ? Include also HEP manpower.
2. How are we protected against the raise of price ?
3. How are we protected against licencing changes ?
4. What efford would be needed to develop it ourselves? In Atlas ? In HEP ? Try to guess the equivalent price.
4. What quality is it ? Is it state-of-the-art of the current software ? How does it compare to similar packages which are available ?
1. Does it use standards  ? Whatever is meant as "standard". Does it use any nonstandard feature which is standardized otherwise ? If so, is it justified ?
2. Does it use Open or Proprietary solutions ? Is proprietary solution justified (if used) ?
3. Is it well encapsulated ? Is it not too intrusive ? Should anything be changed  in the rest of the Atlas software to accommodate the package ? Will anything have to be changed if we want in future to withdraw the package ?
5. Does it run on platforms and environments supported by Atlas ? If not, will it run ? When ? And for which price ?
1. How fast will be port to new versions of OS, compilers,... ?
6. Are there any Atlas-wide or HEP-wide constraines concerning it ?
1. Is it used by other packages already used by Atlas ? So we are alreday using it anyway ?
2. Does it clash with anything already used by Atlas ? Are we constrained by another (alreday made) choice ? Could we change the other choice ?
3. Does anything alreday used by Atlas offer the same functionality ? Better or worse ? Is it possible to move ?
4. Is there anything else offering the same functionality inside HEP ? Better or worse ? Are we (will we be) constrained by the other package ?
7. Which level of support is available ? HEP, external, no at all ? Does the maintainer officially exist ?
1. How fast will be bug hunting ? Is there official procedure or just "good will" ?
2. May we request package enhancements ? Is the maintainer willing to enhance package according to our  requests ? For which price ?
3. May we request package changes ? Is the maintainer willing to make the changes which would modify the package (so other customers may feel unhappy) ? For which price ?
4. May we request port to other platforms and environments ? Or are we depended on the maintainer choice ? Which is the price ?
5. How is future support insured ? What will happen if current maintainer stops existing.
6. Are we the only customers ? Is the package developed for HEP ? For Atlas ? For scientific world ?
7. Is HEP interesting for the vendor/maintainer ? What percentage of customers are we ? Judge both the money and the importance.
8. What is previous experiance with the vendor/maintainer ? Inside Atlas ? Inside HEP ? Inside academic world ?
9. How big the producer is ?
10. Is it alreday used in HEP ? With which satisfaction ?
8. Is documentation adequate ? For both developers and end-users ?
1. Is there documentation widely available ? Is it in machine readable form. Can we print it ?
2. Is there HEP specific documentation ?
3. Is API well documented ?
9. Is source available ?
1. Will source be available in case of end of support ?
2. Will source be available in case of maintainer bankruptcy ?
10. Is design available ?
1. Is design available in our supported format ?
11. Is test suite available ?
1. Can we extend it ?
2. Can we include it in our testing schema ?
3. Will it pass our existing tests ? If it substitutes (part of) our existing package. Maybe test should be a little modified (when interface changes) ?
12. Can it be efficiently used in our SRT schema ?
1. Will it reside inside or outside of SRT ? If inside, should we do anything after each package upgrade ourselves to put it into SRT ? If not, how will we handle it ?