Project 12: Hardware features Toronto 1978: ============= Professor Hull presented a paper entitled 'Desirable floating-point arithmetic and elementary functions for numerical computation'. The paper summarized proposed specifications for floating point arithmetic and elementary functions. The approach adopted was at the programming level. He was primarily concerned with specifications which were simple, complete and flexible. The impact of numerical specialists on hardware design and computing system construction had been far from satisfactory in the past. Professor Hull said the ideal outcome of present discussions would be the emergence of a generally agreed set of specifications which would eventually be adopted by computer manufacturers. The principal points of the proposals presented were: base - 10 precision - for variables; user declarations such as FLOAT (3,12)x where x would have a 3-digit exponent and a 12-digit fraction. For computation; precision blocks' such as BEGIN PREC(3) - the precision to be exact, not 'at least' as in PL/I. representation - sign-magnitude, normalised. arithmetic - properly rounded, left-to-right evaluation (or some other 'obvious' rule) basic functions - abs etc available elementary functions - emphasis on simplicity. error handling - PL/I-like ON conditions fixed point - is INTEGER necessary? Discussion: Dr. Smith queried the proposed absence of integer variables. Professor Hull replied that a specific function for integer overflow, for example might be necessary, He accepted Professor Fosdick's surmise that an optimising compiler would be allowed to optimise code so long as the effect was numerically identical to evaluation in the proposed standard way. Dr. Reinsch, in questioning the pursuit of simplicity, felt that the provision of functions such as log would give some difficulty. Professor Hull responded by suggesting a simple, general log function and a special one for particular requirements. --- A paper by Dr. Reinsch called 'Principles and Preferences for Computer Arithmetic' was presented. Dr. Reinsch described the background to the paper and acknowledged the contributions of others. He would concentrate attention on hardware (or firmware) features rather than software. Originally he had envisaged producing a grading scheme for various possible features but he was now in favour of stating desired principles and then examining the consequences of those statements (e.g. leading to the choice of base). The principle sections of Dr. Reinsch's paper were: - introduction; largely based on Professor Dekker's earlier IFIP working papers - floating point number system; here the extrema and the resolution factor were introduced. The system was to be 1) Symmetric 2) Balanced 3) contained There would also be special operands for indefinite and infinity results. As for the choice of base, Dr. Reinsch did not feel that any one choice was superior in all respects to other choices. basic arithmetic operations; in this section a number of desirable properties were propounded concerning rounding and basic arithmetic operations. The importance of comprehensive and correct documentation describing the implementation of these rules in a particular system was stressed. - underflow/overflow; any U/O event must always be indicated to the programmer who could then respond accordingly. - relations; these should be exact. - integer arithmetic; a number of principles were presented. conversions; input/output transfers and precision conversions should be exact where possible. Discussion: Dr. Battiste welcomed what he regarded as the 'top-down' view of Professor Hull and the 'bottom-up' approach of Dr. Reinsch. Both approaches were necessary to achieve desired goals. He referred to a problem of 'wrap-around' recently encountered in the production of mathematical software. Dr. Reinsch believed that in suck a case, his minimum requirement was to warn the user and permit recovery in a reasonable way if possible. Professor Hull detected a difference in emphasis in the two approaches perhaps because the term 'user' was being used to refer to different kinds of computer users. Dr. Reinsch's proposed scheme was supported by Professor Dekker. Another aspect of computation pointed out by Dr. Battiste was the need of the statistician for a uniform resolution throughout the whole range. Providing such a facility on discrete- arithmetic machines would always be a problem, said Dr. Reinsch. A further presentation on machine arithmetic was presented by Dr. Smith. It was based on a paper by Kahan, Stone and Coonen that he had recently seen. The paper described the current state of an IEEE-sponsored activity to propose manufacturer-independent hardware standards. The current proposals bore some relationship to equipment about to be released by INTEL though now had evolved somewhat further apparently. (Copies of the paper were distributed to members). The main topics discussed in the paper were arithmetic storage, operations, comparisons and exception handling. Three types of entity in the closed number system were introduced; normalised numbers, infinites and 'nans' (_not a number). The paper went into considerable detail as to how these entities should be manipulated. Computations were assumed to take place in an internal extended precision units. If the result of a computation overflowed, that was to be detected when the result was transfered to arithmetic registers accessible to the user. The recommended number representation was sign-magnitude and the suggested number composition was a 23-bit binary mantissa with an 8-bit exponent. (Dr. Smith offered this explanation for such a choice; it arose probably from the advent of the 16-bit bus on mini computers and from the belief that well-formed arithmetic on 32-bit words was often sufficient). Of the several rounding options presented, the most strongly recommended was proper rounding. Two forms of arithmetic would be available, projective and affine. The mod and sqrt functions were to be implemented by hardware. There was provision for a guard bit, a rounding bit and a 'sticky' bit. A status bit would indicate when rounding had taken place (thus permitting exact integer arithmetic?). Finally, the problem of base conversion on input/output was addressed. It was claimed that numbers which were input then output or vice versa would achieve their original value. Discussion: Dr. Reinsch commented that the proposals seemed consistent with previous work by the authors. The Chairman asked members to respect the draft status of the paper before making external reference to it. --- The meeting resumed discussion of the arithmetic systems considered earlier in the day. Dr. Brown gave what he considered to be a simplified but relevant characterisation of the three main strands of thinking: (a) where human-related costs dominated, the tendency was towards ideas such as those of Professor Hull. (b) if computer costs were paramount (as in weather modelling) with specialised architecture such as the Cray-1 emerged. (c) in most cases, however, a compromise developed between user-convenience (crudely speaking, approach (a)) and the 'machine-dominated' approach (b), e.g. as reflected in the Stone, Kahan and Coonen paper. There was a need to develop standards for all three cases. The meeting discussed Dr. Brown's characterisation at length. Dr. Reinsch felt that his proposals addressed topic (c) but considered that there were extremes other than (a) or (b). Professor Dekker said that it may be necessary to have separate (or at least graded) hardware specification standards where options to meet particular circumstances were included. In Dr. Battiste's opinion, questions of implementation of proposed hardware standards (and of convincing manufacturers of the wisdom of such standards) had yet to be addressed. Commenting on the choice of a base, Dr. Brown considered that Brent's work had established a strong case for base 2 with proper rounding. He believed that Professor Hull's proposed exact-precision arithmetic might be costly to implement on present ranges of computers but he supported the ideas of user-control over precision, except that he wished integer arithmetic to be retained. Dr. Brown concluded his informal presentation by suggesting that the INTEL chip, the Cray computer and Professor Hull's proposals were all cause for celebration and optimism. Professor Rice felt that current efforts should be directed largely towards case (c). Those involved in (a) and (b) realised that they had special needs which would take longer to fulfil. The Chairman considered that it might be possible to influence the builders of special computers. He regarded the approach of Professor Kahan et al, and those of Dr. Reinsch and Professor Hull as all addressing the same problem but at different levels. Dr. Brown's ideas added a further dimension still. Professor Hull felt that inconvenient computational systems build on poorly designed hardware had been inflicted on users for too long. He was happy with the gradual identification of higher and lower goals in terms of computational requirements. Dr. Smith expressed support for Professor Hull's aims but said that hardware designers would require more guidance. Dr. Battiste recalled the history of machine construction and observed that the boundaries between hardware and software were becoming increasingly blurred. Professor Dekker and Dr. Reinsch returned to the question of implementing proposals such as those of Professor Hull. Two particular problems that they identified were the dynamic evaluation of exponents and of special functions. --- The meeting resumed discussion of the issue raised by the presentations of Dr. Reinsch and Professor Hull. Professor Rice was keen on publishing the papers given, particularly that of Dr. Reinsch, soon in order to receive comments from a wider audience. Dr. reinsch described the background to his paper; the Oak Brook meeting had been of particular significance. He stressed that his paper was intended to influence the designers of current hardware systems but not to confuse them with too much justification of design principles from higher-level software considerations. Members of the group were asked to convey any further views on the paper as soon as possible. Dr. Reinsch would then submit the revised document via the secretary to members for formal consideration as a working group document. Professor Hull found the two level approach acceptable. If the two were developed in a compatible way, his proposals at the programming level could be based on sound arithmetic at the hardware level. Professor Stetter agreed, saying that the machine-level definition was an essential first step, from which ideas such as Professor Hull's could be more easily implemented. He felt that Professor Hull's paper should be referred to compiler writers. To remove possible contradictions between the two papers, Professor Hull said he was prepared to study both sets of proposals in conjunction. One potential problem, for instance, might be the treatment of error traps; Dr. Reinsch's proposal was necessary but was it sufficient at the higher level? Dr. Reinsch offered to refer to higher level programming issues in the introduction to his paper. A comparison of the two proposals and that of Kahan et al was given by Professor Dekker. Dr. Smith said that the Kahan set of proposals was a good example of a specialised implementation of ideas like those of Dr. Reinsch. Dr. Einarsson reminded the meeting of Lhe current status of the Kahan paper. The meeting then considered the circulation of the two papers for comment as discussion documents. Dr. Reid and Professor Fosdick wanted an open approach during the comment phase. A definite decision on Dr. Reinsch's paper was required soon (at Baden, perhaps) in Professor Rice's opinion to stand by chance of influencing hardware designers. Dr. Einarsson considered that TC2 approval of such a paper may take some time precisely because it did affect hardware. He mentioned the IEEE repository as a possible place to publicise the draft ideas. Another suggestion was to approach appropriate special interest groups in ACM. Dr. Reinsch wanted to have a quick response from the numerical analysis community before actively pursuing the views of hardware designers. It was suggested that the papers might be given at the Baden working conference and offered for presentation at the Liverpool software meeting in Easter 1979. This led to a discussion of the desirability of deviating front the theme of the working conference. Dr. Einarsson summarised the outcome of the overall discussion with Dr. Reinsch's and Professor Hull's papers; - comments from WG 2.5 members should reach the authors by the beginning of July - the two authors were encouraged to publish their papers as discussion documents in SIGNUM (and other journals if appropriate) - they would act as stand-by speakers at the working conference (and their papers might, at the discretion of the editor, be published in the .proceedings). Baden 1978: =========== Dr. Reinsch surveyed the state of his paper. It had been distributed to interested parties and had not met with any objections. Subsequently it had been sent for publication in the SIGNUM Newsletter. Professor Dekker sent a suggestion and an introduction was also added. This paper was mailed to Dr. C. Lawson. Thereafter two other suggestions had been received, one from Professor Gear and one from Professor Rice. The latter had reworded the paper slightly to improve the English and sent this to SIGNUM for the March issue. Other changes were still possible up to the middle of January, Professor Rice said. There was some discussion on the sentences describing the status of the paper within the group since all members did not totally endorse all the viewpoints expressed therein. Professor Hull, for instance, thought that the original wording adequately represented the differences between himself and Professor Reinsch on this subject. He thought it bad, however, that both should publish independently of one another without referring to the others work. The Chairman asked Professor Hull, Dr. Reinsch and Professor Rice to form a sub-committee on this topic. He also enquired about the current state of Professor Hull's paper and was told that it had been re-written and a shortened version appeared in the Pasadena Conference Proceedings. As a result of this the paper had automatically been sent to I.E.E.E. It had not been submitted to SIGNUM. It was suggested that Professor Hull's paper appear in SIGNUM, together with a summary linking together the ideas of Professor Hull and Professor Reinsch. To this end Professor Hull undertook to see if I.E.E.E. would object to this and Professor Rice promised to investigate the summary. It was agreed that Professor Hull and Professor Reinsch should send their papers to the Chairman of TC-2. Professor Kahan's Paper The question was asked by the Chairman, in the light of Professor Kahan's presentation, whether or not WG 2.5 ought to take a stand in the dispute. Both Dr. Reinsch and Professor Stetter thought some action mandatory but, whereas the former thought immediate action premature, the latter thought it necessary. Dr. Brown thought that WG 2.5 could endorse the features common to both proposals but Professor Gear pointed out that this would effectively endorse the DEC proposals. He advised against becoming involved. Professor Fosdick too expressed concern, observing that the Payne proposals had not been given the same hearing as Kahan's, and noting that WG 2.5 had nothing to gain by taking a decision. Dr. Reinsch supported a suggestion by Dr. Ford to try and order machine arithmetic features from 'most preferred' to 'least preferred'. Floating-Point Model Dr. Lawson described the Pasadena Conference, which had 27 speakers, 95 attendees and lasted 2 1/2 days. Members of the X3J3 committee had attended to the first morning's session and Dr. Lawson, Dr. Smith and Dr. Brown all spoke at the meeting. It was his impression that X3J3 were aiming to define a 'core' for the new FORTRAN which would be more of a subset of FORTRAN 77 rather than embracing it. Dr. Ford spoke about the paper, 'Parameterisation of the Environment for Transportable Numerical Software' and this provoked some interchanges between himself and Dr. Brown, who was anxious that his own model be accepted. Dr. Ford wanted to make some modest revisions along the lines indicated by Dr. Cody and promised to bring specific details of these changes to the next meeting. This approach was agreed nem. con. Professor Hull saw no conflict in proceeding with the parameters paper and the Brown-Feldman paper in parallel. Dr. Brown was keen however to submit proposals to X3J3 this year, when he believed the committee would be receptive to these ideas. Dr. Reid introduced the 'Intrinsic Functions' paper. It was a small paper with no major changes from the Toronto meeting. Dr. Brown was unhappy about the paper since it was similar to the Brown-Feldman paper with references to model parameters deleted. Pressed by Dr. Reid he said that he would sooner go this way than not go at all but thought it impossible to adequately define the functions without reference to the model parameters. Dr. Reid was unwilling to publish the paper as an individual and there was some discussion as to how the paper should be introduced. A wording was suggested by Dr. Einarsson, which was subsequently amended by TC-2, to read: "The paper has been discussed by IFIP WG 2.5 on Numerical Software and has been approved by the Working Group, but does not constitute an official IFIP document". The content of the document was subject to discussion. Professor Gear wanted functions to be declared INTRINSIC if they are used as such. Professor Dekker wanted a dynamic definition of EPSILON. Dr. Reid however was anxious to make only minor revisions at this stage and to finally clear the matter up. The changes he proposed were: Page 2, line -3 insert "and k is some constant (e.g. 10)". In the footnote replace "real" by "processor". Delete the last line and insert "No floating-point operation can cause underflow if its result is outside (-sigma, sigma). Page 4 - bottom. Delete "F is setexponent ..." and read X_i = setexponent(X_i, integerexponent(X_i)-K)". Replace "note that ... permissible exponents" by "It is unfortunate that two function calls are involved in the loop, but they are such simple functions that we could expect a compiler to provide in-line code". Dr. Brown asked Dr. Reid to consider six functions, not three. Professors Stetter and Hull thought two papers mutually referencing one another a possibility, but Dr. Smith said.that X3J3 needed help - one proposal not two. The matter was to be discussed in smaller groups outside the meeting that evening. Intrinsic Functions Professor Hull reported that he and six others met the previous evening and as a result the gap between Dr. Brown and Dr. Reid narrowed. The latter would acknowledge the proposal for 3 extra functions and also the different definition of EPSILON. The revised paper would be distributed in early January and remarks on it invited. Should the comments be serious these would halt publication. Dr. Brown gave notice that he intends to submit the Brown-Feldman model for approval at Novosibirsk. He asked that his silence of the Ford and Reid papers not be interpreted as approval nor prejudice the motion at Novosibirsk. Dr. Reid's paper was to be given the same support, and introduced by the same wording, as Dr. Reinsch's. It was to be sent to X3J3. Arithmetic Proposal Professor Gear reported on the findings of a sub-committee meeting the previous evening. They felt that some action was necessary and thought that this should be in the form of an ordered list of features that the group would like. The proposed list was 1) Unbiased rounding, choosing even when equal. 2) 32/64 binary base, with 8/11 bit exponent is acceptable. 3) Traps available for overflow, underflow, division by zero and all other unusual arithmetic conditions (ones in which the true answer cannot be represented as a valid number). It is necessary to have the ability to disable traps. 4) Extended precision and exponent range on a local range is desirable, as is the existence of denormalised numbers. 5) Other types of rounding are highly desirable but unbiased rounding must be the default. 6) There should be representations for +/- infinity and NANs. 7) Representations should be in order - infinity < - NAN < 0 < NAN < infinity. If cost considerations are paramount then ordering, other roundings and extended precisions could be omitted. The list was the subject of some debate. Dr. Brown wanted to leave out the last paragraph but Dr. Ford said that it was important and showed that the group was aware of practical considerations. He was supported in this by Dr. Einarsson and Professor Stetter. Dr. Brown thought that the document should contain a reference to Dr. Reinsch's paper. There was general agreement in principle that there should be a paper, but the details had yet to be resolved. This would enable Dr. Einarsson to ask TC-2 for their support in sending the paper to I.E.E.E. when he attended TC-2 later in the day. Novosibirsk 1979: ================= Brown reported that his paper on the floating-point model had been rewritten (IFIP/WG 2.5 (Novosibirsk-11) 611), that the model has been accepted as a basis for the arithmetic in the language ADA (Ford queried whether this was really so), that his proposal for environment parameters had been revised (IFIP/WG 2.5 (Novosibirsk-10) 610) and that Schryer's test program for checking parameter values has been completed. He drew attention to the preliminary results given in an appendix to (IFIP/WG 2.5 (Novosibirsk-10) showing values for parameters and details of certain apparent anomalies. Fosdick remarked that this table was particularly useful in that it permits ready grasp of the efficacy of different manufacturers' arithmetic. Reinsch, however, remarked that certain other dirty features, such as biased rounding, may pass unnoticed. Schryer's program may need quite long runs (e.g. a few minutes on CDC 7600) for results in which one can be fully confident. Gear remarked that different compilers on the same machine may need different parameter values. Harwell 1980: ============= T.E. Hull: Handling of exceptions and some remarks on the design of arithmetic units (see IFIP/WG 2.5 (Harwell-43) 743 Hull explained how he felt exceptions should be handled at the language level and describes his discussions with hardware experts on the possibilities of designing an efficient decimal unit. The outcome is that the speed of a decimal unit could be very comparable to a good binary unit, but the cost might be double. He hopes to arrange a research project involving the actual construction and experimental use of such a unit. Discussion Curtis: The main difficulty is that we are stuck with architectures based on fixed words (e.g. 32 bits). Hull: You can pack three decimals into 10 bits. Curtis: This may permit a 32-bit word to be used efficiently, but surely the overheads of packing and unpacking would be substantial. --- Principles and preferences for computer arithmetic Rice introduced a draft (IFIP/WG 2.5 (Harwell-14)714) of a paper that briefly summarizes the work of both Hull and Reinsch. Hull was worried by the apparent conflict and remarked that hardware designers do not read SIGNUM newsletters. An outlet to the hardware world is needed. Fosdick said he thought the paper should relate to the Rice IEEE standard; the group should not appear to be "mudding the waters". A subcommittee consisting of Rice (chairman), Hull and Reinsch was set up to prepare a final version for publication, to be presented for approval at the next meeting (draft: IFIP/WG 2.5 (Harwell-38)738). Madison 1982: ============= The "Karlsruhe approach" to arithmetic -------------------------------------- Kulisch explained that all he needed in Fortran 8x to permit the use of his arithmetic was the ability to define his own operators. Hull asked if he did not need additional rounding modes, but Kulisch said that these could be written as assembly routines. Smith asked about Ada; Kulisch said that the problem here is that only existing operators may be overloaded, but Smith felt that this could be circumvented by defining another type with a single component. Gentleman remarked that simple overloading of existing operators did not give rise to parsing problems but was not so useful. Rice asked whether there was any chance of acceptance and Smith said that he thought there was in view of the core plus modules problem. Einarsson asked whether Kulisch should not write a tutorial paper and Kulisch said he was preparing such a paper. Hardware features, exception handling and precision control ----------------------------------------------------------- Hull said he would welcome guidance from members of the group on features for the arithmetic unit CADAC being built at Toronto (see the 29-page document IFIP/WG 2.5(Madison-35)935), copies of which he would be happy to supply to anyone interested. Hull pointed out that the group's terms of reference included agreeing "from a numerical standpoint on a set of hardware and software features for a computing system", but that we had not always been very successful (e.g. the attempts by Rice to reconcile the views of Hull and Reinsch on desirable hardware). What should the group do now? He felt it should aim to produce agreed documents about "ideals and principles°' re i) exception handling and ii) precision control. On exception handling, Hull thought that the user should be able to specify the action to be taken following an exception and should be able to set a result (e.g. on underflow, set to zero). He proposed that each handler should be associated with a given scope (perhaps as an internal procedure). He noted that our present preprocessors do not include error handling. On precision control, Hull noted that the most important idea is that of separating the precision of operators from that of variables. He also noted the automatic change of exponent range with change of precision and that Kulisch's accurate inner product can be expressed in his language. Reid mentioned the current X3J3 activity on exception handling by a group of Europeans. He thought that it would be very desirable for the group to be taking an interest in this. Dekker felt that exception handling and precision control were separate topics and that exception handling is really needed. Soederkoeping 1983: =================== The "Karlsruhe approach" or guaranteed accuracy arithmetic. ----------------------------------------------------------- As Kulisch was not present, Stetter gave a short report. After reminding the WG members of the main features of the approach, he mentioned the following activities (besides presentations at various places and occasions): A scenario seminar exploring the usability of the approach in various application programming situations ; the completion of a Fortran Subroutine library (containing the algebraic routines and many utilities) to be implemented on a mini with the arithmetic hardware features; further work on applications to analytic problems; consideration of the chaining of high accuracy algorithms. The software tool aspects of the available software were stressed. It was also recalled that the X3J3 committee, at the meeting that Kulisch attended, has been in favor of hooks in the language to enable a module implementing "Karlsruhe" arithmetic to be developed. Documents: ========== 1. W.S. Brown "A realistic model of floating-point computation," in Mathematical Software III, J.R. Rice (ed.), Academic Press, NY, 343-360, 1977. 2. C.H. Reinsch "Principles and preferences for computer arithmetic," SIGNUM Newsletter, Vol. 14 (1), 12-27, 1979. 3. T. E. Hull "Desirable floating-point arithmetic and elementary functions for: numerical computation," SIGNUM Newsletter, Vol. 14 (1), 96-99, 1979. 4. W.J. Cody, "Towards sensible floating-point arithmetic," COMPCON '80, San Francisco, 1980.