Project 8: Criteria for evaluating quadrature, ODE and 
	   integral equation software		
		
Interpreted as covered by the Baden conference on performance
evaluation, see project 18.

-----------------------------------------------------------------------
Oxford 1975: 
============

Special session on evaluation of algorithms summarised in the minutes, pp 4-6.

Position Paper: Building a Library of Numerical Algorithms: A Case Study 
from the Handbook on 'Linear Algebra' - C H Reinsch

The author observed that there are there stages in the production of 
mathematical software: (i) the invention and selection of mathematical 
constructions (methods of computation), (ii) their casting into 
numerical algorithms (uniquely determined sequences of basic operations), 
(iii) the final transition into programs working on an individual computer 
(e.g. a FORTRAN program for an IBM 370 machine).

He felt that the first stage is the domain of individual research so 
that a group devoted to communication could contribute	little in this 
field.	Likewise, he stated his belief that much of the third stage is
too computer-specific to be the target of a group activity, On the other 
hand, it was the author's view that the second point would yield a
more natural objective for the group.

The working group would compile from the literature recommended methods 
of computation and evolve each of them into the form of a numerical 
algorithm as defined above and so gradually build up a universal library 
of such algorithms.

The paper then discussed various premises of such an enterprise based 
on the experience gained in the work on the Springer Handbook of 
Automatic Computation, Volume II: Linear Algebra (eds. J. H. Wilkinson 
and C. H. Reinsch).

Discussion:

There was agreement with Dr Wilkinson that the divisions in the papers 
in the Handbook were a good idea.

The respective merits of Algol 60 and FORTRAN were argued and Dr Lawson 
remarked that in some ways languages such as BABEL represented a gradual 
drawing together of opposing viewpoints. Pursuing Dr Reinsch's
theme that algorithms should be written in an Algol 60 subset to enable 
them to be translated easily into other languages, especially FORTRAN, 
Dr Smith enquired how dynamic arrays could be translated. He felt
that it was wrong to expect people, unfamiliar with the algorithm, to be 
able to deal adequately with difficulties that may arise in different 
machine implementations.

A theme, which was to recur throughout the afternoon, was introduced by 
Professor Stetter who wondered whether the group was primarily concerned 
with software or algorithms. He and Dr Lawson favoured the Handbook 
approach, and welcomed EISPACK, which sets standards for others to 
follow in the creation of libraries. Professor Stetter emphasised
the need for means of evaluating an algorithm and mentioned the work of 
Professor Hull's group for ordinary differential equations.

At this point the Chairman conveyed a message from Professor Hull which 
explained his absence from the meeting because of sickness.

Professor Hull had volunteered to -

(i) Co-ordinate the creation of a set of test problems for stiff and 
non-stiff problems in ordinary differential equations and make 
them available on magnetic tape.

(ii) Attempt to categorise these problems.

(iii) Supplement each problem with a routine capable of solving the 
problem within specified requirements.

The meeting adopted Dr Ford's suggestion that he should write to 
Professor Hull expressing regret at his absence due to sickness.

The group then discussed possible objectives. Dr Reid felt that the 
creation of a collection of a set of machine readable test problems was 
most important and he was supported in this by Dr Ford.	Professor Stetter 
suggested that the group consider the qualities required of a good algorithm. 
He felt that a survey of user requirements would be useful and observed
that a user should not have to choose between algorithms, but have this 
done automatically for him by the computer, given details of his 
particular problem.

Discussion of the criteria for evaluation of numerical software ranged 
through the choice of algorithm, performance, documentation and user 
convenience of software, the tools for the evaluation of software and 
the collection of statistics on user needs. Professor Stetter felt 
that more progress would be made by concentrating the Group's attention 
on specific subject areas and quadrature, ordinary differential equations 
and integral equations were mentioned as candidates for consideration. 
Of them quadrature appeared most favourable as a first step.

Amsterdam 1977:
===============

Dr Smith and Professor Stetter are separately working on this and
the conference next year will address this problem.