Numbering Peano

Republishing before Silence

2010-04-30T21:17:00.001-07:00

The nfoCentrale blogs, including Numbering Peano, were published through Blogger via FTP transfer to my web sites. That service is ending.

As part of the migration, I am republishing this blog in the latest stable template format.

Then there will be silence as Blogger is unhooked, although the pages will remain.

No new posts or comments will work until I updated the web site to use its own blog engine. Once that migration is completed, posting will resume here, with details about what to know about the transition and any breakage that remains to be repaired.

Meanwhile, if you are curious to watch how this works out, check on Spanner Wingnut’s Muddleware Lab. It may be in various stages of disrepair, but that blog will come under new custodianship first.

Friday Miser Note: Interchanging Code

2009-11-13T09:34:00.001-08:00

Technorati Tags: Miser Project, oMiser, XML, persistent object interchange

While I think and post mainly about the theoretical abstractions that are important in getting Miser and the Frugal language done properly, I am also thinking ahead to the way the Miser engine can be delivered as a platform component (Windows DLL, .NET assembly, etc.) that hosts Miser on behalf of a software application. I have in mind that any kind of Frugalese user interface for a console application, including any interactive development environment (IDE), would be built atop a hosted oMiser engine as an important first case.

Because the Miser model is platform independent, it would be nice to have a platform-independent way to save, load, and interchange Miser “codes” in a neutral, highly-interchangeable format.

XML comes to mind. Although XML is not a compact binary format, it is a well-known, standard format for which a great number of problems are already solved, including versioning, the ability to deal with extensions gracefully, the complete hiding of the storage structure, and harmony with my intention that the Miser abstraction be universally comprehended. (That is to say, it shall seem as if there is only one oMiser in the world, and the fact that implementations are distributed and replicated is as invisible as can be practically achieved.) In the oMiser case, the XML stream would also be highly compressible using widely-available compression techniques.

The interchangeable “object code” among all oMiser implementations will be standardized in an XML stream.

I propose to abuse the purity of XML for this purpose. The XML will be well-formed but it won’t be the way XML would be naturally used to communicate a persistent format for oMiser Obs.

My basic plan is that the loadable Ob will be coded in a reverse-Polish notation conveyed in XML. The notation will have a syntactic structure that conforms to a BNF grammar such as

〈ob〉::= 〈term〉 | 〈ob〉〈un-op〉 | 〈ob〉〈ob〉〈bin-op〉

allowing the bottom-up build-up of an internal Ob representation as the XML stream is being read. Not every Ob operation needs to be implemented, only those needed for constructing constant Obs:

Here 〈term〉 entries will specify individuals or links to Obs defined earlier in the stream.
There will be unary〈un-op〉operations, including one for ob.e(ob) and one for labeling an Ob so that it can be referenced later in the stream. Linking to already-expressed Obs is indispensible for simplifying repetitive introduction of the same Ob in the interior of a larger Ob. I intend to make use of xml:id for this purpose.
There will be binary 〈bin-op〉operations, including one for ob.c(ob, ob).
There will be additional mark-up, but this is the essential structure.

XML streams for the loadable format will be supported at the core level of an oMiser implementation. Given a handle on an Ob, there is an interface that will emit the XML stream for persistent representation of that Ob for any interchange purpose. Likewise, there is a low-level interface on every oMiser engine by which an XML stream is accepted and a handle for the corresponding Ob returned for use in the hosting application.

The XML representation will be flat. That is, the <oMiser:ob> XML element contains a flat sequence of <term />, <un-op />, and <bin-op /> elements. It is easy to process such a flat structure, and it is easy to ensure that such a structure is well-formed according to the grammar for〈ob〉given above.

There are three interesting challenges for this structure. These need to be explored to ensure that the choice of a reverse-polish Ob-loading is as practical as I intend for it to be:

It is more difficult to produce the loadable format than it is to consume it. This desirable trade-off has to be a containable problem. In particular, we need storage-efficient techniques for emitting the bottom-up reverse-polish construction of any Ob that we are given a handle to.
Because Obs are inherently immutable, sharing of common parts is an important storage economizer in oMiser implementations. Taking advantage of opportunities for sharing of substructure in a running implementation is an interesting challenge. Preserving or even enhancing sharing in an exported loader format is an additional opportunity. (It may well be the case that improved sharing and the consequent logical compression of the persistent form is better done by a separate post-processing optimization utility.)
While the run-time validation of a well-formed reverse-Polish sequence of elements is trivial, its expression as a constraint on an XML schema is not so easy. We don’t need an XML schema to implement a consumer of the format (and we can use namespace and DTD rules to signify that the format constraints apply). Just the same, it is valuable to have some sort of schema that expresses the reverse-Polish constraint on the sequence of content elements within an XML <oMiser:ob> element.

I was reminded of all of this when I recently read the 2009-09-29 Rick Jelliffe post on context-free XML. The shoe-horning of the context-free reverse-Polish flat stream for〈ob〉as content for an XML <oMiser:ob> element creates a context-sensitive problem for expression in some sort of XML schema. This post is intended to elaborate on the comment that Rick’s post inspired from me. I trust this is useful context for that. (As usual, your mileage may vary.)

Discussing oMiser implementation considerations is long overdue in any case.

Alan Turing Birthday: July 23, Centenary: 2012

2009-06-23T08:22:00.001-07:00

Technorati Tags: Alan Turing, Turing Machine, Turing Test, Bletchley Park, Turing Centenary, computation theory

I just learned that July 23, 1912 is Alan Turing’s birth date.

There will be a centenary celebration throughout 2012 with UK events held in Cambridge, Manchester, and Bletchley Park. There are already many interesting links on the site and among the Turing Centenary Advisory Committee.

Bletchley Park, where the work on cryptography was instrumental in the conduct of World War II, has fallen into disrepair and neglect. One hopes that renewed attention on the life of Alan Turing may bolster support through 2012 and beyond.

[I’m thinking maybe this is an interesting way to finally meet-up with Charles Petzold. I might even have more to offer on insights about computation that I have gained from the Miser Project by then. This year is the time to renew my attention on that.]

Miser: Interpretations of Identity

2008-06-16T11:54:00.001-07:00

Technorati Tags: formalized theories, interpretations of theories, FOL=, first-order logic, equational identities, identification, distinction

[update 2008-06-17T16:08Z: I added t0' because I couldn't stand not to show the better refactoring of t0. I added mention of equivalence classes in an interpretation and also came up with conditions other than invalidity for finding an interpretation unusable.]

In The Logic of Ot, I said that I would use informal expressions of Ot, the logical theory that applies to Miser Obs. Now that there has been some use of the special characters and notations of First-Order Logic with equality, I want to take advantage of that to talk about interpretations of identity in models of Ot. The ability to identify and distinguish has great bearing on computational systems, and identity as an interpretation is particularly useful to explore.

= as Equivalence Relation

With FOL=, identity and the relational operator, "=", are taken as given, and the following hold:

=1: ∀x(x = x)

=2: ∀x∀y(x = y → y = x)

=3: ∀x∀y∀z(x = y ∧ y = z → x = z)

≠0: ∀x∀y(x ≠ y ↔ ¬(x = y))

The first three are the common properties of equivalence relationships: "=" is reflexive (=1), symmetrical (=2), and transitive (=3). The final condition is essentially the definition of "≠" in terms of "=".

What Are We Talking About?

In a first-order logical theory, the variables (x, y, and z as seen in ∀x∀y∀z) are understood to refer to objects in the domain of discourse. We only know what there is to know about that domain from the introduction of constants and expression of conditions that are theoretically required to be satisfied over that domain. For FOL=, we are given an equivalence relation (expressed with the symbol "="), which tells us very few things about conditions under which variables can be taken as referring to the same object of the domain of discourse.

It should be apparent that having "=" doesn't tell us much about the theoretical objects, although it is more than not having "=" (and its partner, "≠").

An intended interpretation could well be that objects in the domain of discourse be identifiable (we can tell when we are referring to the same one) and discernable (we can tell when we aren't). Let's see how that might work.

A Tiny Domain

To provide some practice with ideas of practical interpretation, consider the logical theory obtained by adding the following conditions:

t0: ∃x∃y∃z∀u( x ≠ y ∧ y ≠ z ∧ x ≠ z ∧ (u = x ∨ u = y ∨ u = z) )

or the equivalent,

t0': ∃x∃y∃z( x ≠ y ∧ y ≠ z ∧ x ≠ z ∧ ∀u(u = x ∨ u = y ∨ u = z) )

The intended reading of this is as

There are at least three different objects in the domain of discourse, and
Any object in the domain of discourse is one of those

which is to say, there are exactly three objects in the domain of discourse.

If we only have that one additional condition (t0) in our logical theory, we know nothing beyond that.

Notice that we have not labeled the three theoretical objects in any way. All we have provided for is that there be exactly three.

It will be useful to appear to be more specific by naming them:

t1: A ≠ B ∧ B ≠ C ∧ A ≠ C

t2: ∀u(u = A ∨ u = B ∨ u = C)

Here, A, B, and C are constants for objects in the domain of discourse. We haven't provided much more than what (t0) assures us of, although if there is more to say about the ways that the three objects differ, having the constants to refer to may be useful.

Illustrative Interpretations

The first interpretation will be in terms of numbers. Assume the system of numbers and arithmetic. Take that system as separate from the logical theory consisting of FOL= plus t0 and t1. (t2 is actually a consequence of that much.)

One interpretation of our tiny theory in number theory would be by saying that the interpretation of A is any n < 0, the interpretation of B is 0, and the interpretation of C is any n > 0. We could be specific, say, with interpretation of A as -1, B as 0, and C as +1. We could also say that A is all n < 0, B is 0 only, and C is all n > 0. That is, A, B, and C correspond to distinct classes. Since they have no members in common, these are known as equivalence classes. We'll explore that further when we return to exploration of Ot.

It doesn't matter, here, how the interpretation is chosen, so long as, having made it, we stick to it. The system in which we make the interpretation is a model (in the loose sense of Reality is the Model) provided that all deductions in our logical theory hold in the interpretation.

Because the logical theory says nothing about aspects of the model that are not accounted for in the logical theory, those matters are irrelevant to the conditions of the logical theory. It does not matter how many different ways the interpretation could be made in the model, so long as when one is made, the logical theory is seen to hold for the interpretation. There is a common fallacy involving reasoning about extra-theoretical characteristics of the model to argue that the theory is incorrect or inapplicable, when the disagreement is more-appropriately viewed as one over choice of interpretation. It helps to carefully separate the theory from its interpretations and models to avoid that pitfall. The abstract theory and the logical formalism is helpful in that regard, even if it feels quite unnatural.

"="/"≠" Are Interpreted Too

It is easy to overlook one important feature of an interpretation of our tiny theory: There must be an interpretation for "="/"≠" in the model as well. That comes along too easily in our choice of interpretations in the system of numbers and arithmetic and it is easy to overlook. When we dig into computational systems and the details of Miser, the ability to discriminate "="/"≠" in particular interpretations becomes very important.

Finally, as an interpretation in reality: let A be earth, B be wind, and C be fire. This is a questionable interpretation quite apart from the omission of water. The difficulty is assuring that these are cleanly distinguishable concepts. What do we do with flaming molten lava and the sucking wind of a forest fire?. We will stumble here at least in an effort to have well-determined "="/"≠" and understandable communication of the conditions that others can accept and apply. The simple, practical conclusion may be that the interpretation is invalid (or simply meaningless/useless) and the theory is inapplicable in that case.

In other cases, a certain conceptual sloppiness, if carefully circumscribed, may be tolerable in having useful interpretations in reality. It remains to be seen whether that is ever very workable.

Turing Arrives: Petzold on Arithmetic

2008-06-14T11:00:00.001-07:00

Technorati Tags: Charles Petzold, The Annotated Turing, Alan Turing, Andrew Hodges, Keith Devlin, Mathematical Realism

previously:
2008-05-26: Catching Up with Turing
2007-11-22: More Annotated Turing
2007-11-04: Petzold Annotates Turing!

related:
2007-11-17: Different (Universal) Computation Models
2008-02-29: Miser: Frugalese for Applicative Operations
2008-05-06: Miser: The Logic of Ot
2008-05-29: Reality Is the Model

I pre-ordered my copy of Charles Petzold's The Annotated Turing on November 22, 2007. On May 24, I had to authorize a delay in the estimated ship date. I waited patiently. When I saw that Petzold had his copies, I wondered if my order would fall through the cracks. The amazon.com site listed the book as in stock, but I had no word. That was resolved this Monday, June 9, when I received notice of shipment. The book arrived two days later by postal mail.

The problem with actually having The Annotated Turing in my possession is deciding when to start and clearing the time to do it. I did start reading at the end of the book, and I have nosed into a few other sections. Naturally, the book arrived at a moment when all of my projects are behind and I am already starting an important new one. A systematic reading is yet to come. I know I will love it if only for the historical threads and connections that Petzold traces in the book.

As part of the tracing of connections, Petzold has been reading One to Nine by Turing's biographer, Andrew Hodges. There are numerous connections traced there, and I like it that Petzold finds himself arguing with Hodges as he works through the book.

Yesterday, Petzold comments on Hodges' objection to memorization of arithmetic with recognition of his own experience in learning the multiplication tables. The interesting idiosyncrasy is how Petzold failed to have automatic memory of certain multiplication combinations and he would solve those cases by algebraic deduction when needed. That resonated for me. There are many cases where I did not remember a rule, but I could and did recreate it on demand. I also share Petzold's having done that long after simply memorizing the result would have been more productive. (This shows up in other activities of mine too, including re-inspection of already-written code to remind myself that it is sound and what the context is before adding more to it.)

I wonder how much this ability to have abstracted an applicable principle (in my case, remembering the times-11 and times-12 cases in terms of times-10 plus times-1 or times-2) leads to algebraic facility and the handy use of identities and mathematical induction well before I developed anything like a fundamental understanding of number theory over the course of my adult years. I recall re-derivation as being valuable in test-taking and yet it is not as direct as having embodied the result for immediate availability.

I can't tell you how many times I have verified for myself what the correct formula for the sum of the first n integers is by redoing the constructive derivation. My doubt is always between n(n+1)/2 and n(n-1)/2 and it is, of course [easy for me to say], the former. I say that not because I have memorized it but because I know how to tell quickly another way, the first way I ever saw it "proved." I suspect that I have just sped that up for myself by looking at it anew this time. Then there's the one about the sum of the first n powers of 2 and what it looks like in binary, etc. I suspect that our diminished respect for the teaching of arithmetic and how to verify arithmetic results is causing trouble for students and their teachers when it is time to approach algebra where one can't avoid dealing with ratios and fractions by using a calculator.

Seeing this latest post from Petzold has me thinking of the connection with a recent paper by Keith Devlin (via Richard Zack), "The Useful and Reliable Illusion of Reality in Mathematics." Two connections that come to mind: how we might come to exercise our capacity for abstract, conceptual thinking as we develop our facility with language, and the tendency to see mathematical conceptions as real. In the second case, Petzold has observed that Turing's machine was his idea for a "real" computer, and I am surprised by that. There are deeper connections in the Devlin paper with how we end up regarding mathematical objects, and that is worthy of separate discussion with regard to what makes computers so successful and so devilishly difficult to deal with.

Reality Is the Model

2008-05-29T14:46:00.001-07:00

Technorati Tags: physics, formalized theories, interpretations of theories, models of theories, reductionism, empiricism

[cross-posted to Orcmid's Lair. This is at a level of abstract speculation that is more appropriate here than there. However, I would like a broader audience, and reactions, to what strikes me as having practical importance in how we develop successful computer-based systems.]

During my regular Tuesday buddy call with colleague Bill Anderson, it suddenly occurred to me that I could account for reductionism, an error that scientists and others (software technologists and their masters, for example) make. It is all captured in the following statement:

Theories don't model reality; reality is the model.

I don't recall what prompted my exclamation on the subject during the call, unless it was something about how objectionable "code is the model," "code rules," and other developer slogans are, where the implementation of something becomes the specification, denying us access to any useful answer to the question "implementation of what?"

Now, it is not a new thing for Bill and I to be discussing these issues, including this view of the role of theory. What hadn't landed so sharply was how viewing theories as models for reality is the very pitfall that engenders reductionism.

There's much more careful development required as part of arguing for the usefulness of "reality is the model." I have been looking at that in a setting where I am conducting a theory-driven implementation of some software in the Miser Project. Bill and I discuss this even more where it matters a lot to information technology, in contrasting What Computers Know with what Programmers Know to Do and how there is an essential gap between how computer systems are built, the requirements that those systems are meant to satisfy, and the world of opportunity in which those systems are instruments of human purpose.

For now, I want to look at the statement in the context of how we appear to arrive at theories and then apply them as given.

A word of warning: the value of "reality is the model" is not that it is "true." The value to be found is in having a more useful and powerful way of looking at what we do with theories in contrast with the limitations of imagining our theories to be modeling reality.

Where Theories Come From

There seems little doubt that theories started out as explanations of the regularity in our experience of reality, of the world. Some of these theories were, and still are, very pre-scientific (as theories about theorizing might be as well, and that won't stop me).

At some point in the course of the scientific revolution, say around 1600, typified by the work of Frances Bacon, there was an important move to development of scientific theories via inductive generalization from observations of nature, not deduction from some principles of cause. The reliance on experimental confirmation and empirical observation became important. A consistent case of contradictory results could show where the theory is inapplicable or even completely incorrect. One risk is that expression of a generalized (abstracted) theory might be taken as an explanation of the nature of nature as in "objects at rest tend to remain at rest."

Emergence of mathematical sciences, illustrated in the achievements of Isaac Newton, had a profound impact. It permitted the deduction of consequences by calculation or proof, and it permitted the experimental confirmation of those deductions by natural experiments. Notice, however, that the deduction occurs inside the theory, as it were, and the correspondence of the conclusion with reality is an empirical matter.

Theories on Their Own

The mathematical formulation of important theories, and the computational applications of those theories, are removed from reality. Once we are operating in the formal, mathematical theory of a science, there is no reality there. The connection to the reality is accomplished by our interpretation of the mathematical theory as being about reality. Being about something, especially reality, is not a feature of mathematical systems. Being about something is how we interpret results in the mathematical formulation as applying to the world in accord with a scientific thesis. In other words, the scientific thesis part is not expressed in the mathematical formulation. That is what we add ourselves (even though it is what led us to the formulation and why it might be of any value to us). Some of these interpretations have been so reliable and so useful, we tend to speak of the expressions of the theories as laws (E = mc² being a popular one, force being proportional to rate of change of momentum being less familiar, although we experience its confirmation every day).

There is a combination of deductive process (predicting via calculation, say) and inductive formulation in this approach. We might say that (empirical) experience has shown that the interpreted deductions are reliable and that the theory is a good one in that sense.

The pitfall is to think of the theory as the truth, as somehow explaining how it is that the interpretation of findings in the theory align. Perhaps the most current conceit of this nature has to do with confusion of what nature does as computation because computational processes have some similar characteristics.

Interpreting Theories and the Reductionism Pitfall

There is an area of mathematics called "model theory" or what, here, we might call the model-theoretic view of mathematically-expressed theories.

In model theory, the idea is that a mathematical theory, expressed in a formal, logical way, is given an interpretation by identifying its mathematical elements with those in some other system (usually some other kind of mathematical one). If the interpretation is such that deductions in the first theory have results that are true in the interpretation, we say that the interpretation is valid, and that the interpretation is a model of the theory. The model satisfies the theory. I am omitting many technicalities (and probably abusing model theory) in order to appropriate the basic idea for application to interpretations in the world in mathematical sciences.

An important feature of this view is that the theory need not account for everything in the model. For the model to be a model everything that is true of the theory is true in the interpretation, but the model is not otherwise constrained. The interpretation is only for that aspect of the model that corresponds to the theory. There may many other features and aspects to the model that are simply not captured by the interpretation (and hence the theory). Under the particular interpretation, at least, however valid (in either a mathematical or an empirical sense, as the case might be), the theory has nothing to offer about those other matters. In particular, we are free from concluding that the theory explains the model or dictates its "working." We are also free, in the case of the world and many mathematical and logical theories, of having quite different interpretations in reality for models of the same theory. (Interpreting objects and phenomena as numbers is something we are able to do in innumerable ways. I had to say that.)

The reductionist pitfall is treating the theory as the model (and therefore comprehensive), and claiming the theory to be "about" the world. In that case, there is no way to countenance there being anything else about the world and even the obvious becomes inaccessible. There's some other kind of pitfall in faulting a theory for not embracing all of reality in its interpretations, but I am less concerned about that, although "reality is the model" avoids that too.

Computational Manifestations of Theories

This apparently-backward way of looking at theories is about the application of theories in ways that are useful in approaching reality. The perspective is also contrary to using computation as embodiments of theories and seeing them as somehow modeling the world. Theories may have computational models (as interpretations). This doesn't make the computation model a model of the world any more than the theory is, in this view. I say that there is a computation model of the theory, and there may be an intended interpretation in the world that is a model, but that does not make the computational model a model of the world any more than the theory is.

I find this a very fruitful way to look at a variety of aspects of information technology as it is developed and used. My continuing duty is to articulate this value in less-abstract and directly valuable terms. One curiosity is how this view can still allow for the notion that the act of programming a computer is a case of theory building.

Catching Up with Turing

2008-05-26T20:17:00.001-07:00

Technorati Tags: Charles Petzold, Alan Turing, Turing Machine, Universal Turing Machine, UTM, Church-Turing Thesis, Computation Theory, On Computable Numbers, Mathematical Platonism, Brouwer, infinities

[update 2008-05-29T16:15Z: Added Petzold's 2008-05-28 post with its nice tie-in of logic and Turing.]

update 2008-05-27T16:03Z: A little tweaking in my comment on Petzold's 2008-05-25 essay.]

I received an amazon.com e-mail warning announcing that Charles Petzold's The Annotated Turing is delayed and I needed to approve the shipping delay from May 23 to June 23. A number of times, I will receive one of these announcements only to have it followed by my order shipping immediately. Since Petzold's site reports the book is scheduled to launch on June 16, I will be content to wait.

Meanwhile, Petzold has been posting interesting tid-bits that attracted his attention while working on the book. In many cases, there will be deeper coverage when the text is available. Here are some that I am particularly keen to dig further into:

2008-05-28: Babies Are Illogical: The "Lost" "Chapter" of "Code": Starting out with a look at logical puzzles, Petzold looks at questions about truth, how connected to logic, and related challenges that did not make it into his book, Code: The Hidden Language of Computer Hardware and Software (recommended). In this essay, Petzold illustrates Charles Dodgson's 1896 Symbolic Logic approach using George Boole's original 1854 notation and will tie it back to computability in Chapter 12 of The Annotated Turing.
2008-05-25: Reading Brian Rotman's "Ad Infinitum...": There is an interesting tension between computations that do not stop but in the limit ( as we were inclined to say when I was a school student) the output ~~approaches~~ converges to some computable real number. Turing allowed such cases, and then demonstrated that there are still far more real numbers that cannot be computed than those that can.   The way this is arrived at leaves some questions about how many real numbers we think there should be and whether we are mistaken in where conventional set theory takes us in that regard.
   Although I am not ready to concede to Petzold that any kind of Platonic commitment is necessary in conception of the transfinite, I definitely find this accompanying statement worthy of careful appraisal:
   "We like to pretend that mathematics is the most 'objective' and least human-bound intellectual endeavor, but our view of the natural numbers reveals mathematics to be founded on a very human metaphysical conceit. The natural numbers are not, in fact, "natural" — that is, intrinsically part of nature — but arise out of human discourse."
   In this essay, Petzold explores Brian Rotman's effort to avoid conception of the completed natural numbers (and certainly, in that case, the completed reals). The focus is on tying Turing's work to philosophical issues regarding the foundation of mathematics, an unexpected connection and one that Turing did not explore over-much.
2008-05-22: Turing and Brouwer: The Unexplored Connection: Not expecting to find a connection between the work of Turing and L.E.J Brouwer, Petzold was startled to find one in the short Correction that Turing made to his original paper. Accounting for this leads to a chapter on "Conceiving the Continuum" in The Annotated Turing and this delightful essay on how that all unfolded. This is more on the issue of infinitudes.
2008-05-18: Turing Machines that Run Forever: From the beginning, Petzold has intimated that he wants to illuminate the Turing Machine as Alan Turing conceived of it, with it being acceptable for the machine to run forever. This is not the ordinary formulation that has survived into contemporary computation theory. Turing wants to bring the full Turing characterization to our attention. This essay motivates the difference and the historical context at the time of Turing's formulation and the subsequent revisions.
2008-05-10: The 300 Page Ideal: Considering some of the later blog posts, it is amusing to see this use of "ideal." This is a wonderful essay on the quirky fascination with attempting to produce a 300-page book, since Petzold has never managed to keep one that short.
2008-05-09: Letting Go of the Book: This is a touching essay on The Annotated Turing now being on its way to print, with no more opportunities for changes or misgivings. We are reminded that Petzold had been working on the book since 1999.
2008-03-27: "The Annotated Turing" Typographical Triumph: Petzold provides an account of what he went through to reset the Turing paper so that he could match Turing's symbols and typeface when elements of the Turing paper are discussed in the "annotations." This is something that Donald Knuth will love, since the desired result was obtained (as the editor confides in a comment) by using LaTeX. It is delightful how the reset Turing paper was proofed against the original printing.
2008-03-01: Of a Book Entitled "Mr. Turing's Computing Machine": Petzold provide an account of his 1999 inspiration to produce The Annotated Turing, and the project's early life.

Miser: The Logic of Ot

2008-05-06T19:58:00.001-07:00

Technorati Tags: orcmid, Miser, computation models, theoretical structures, FOL=, first-order logic, equational identities, first-order theory

In the narrow sense, logic is the theory of valid arguments
or the theory of deductive inference.
A slightly broader sense includes the theory of definition. ...
-- Patrick Suppes, preface to Introduction to Logic [5:p.xiv]

Miser demonstrates a model of computation. The specification of Miser establishes the functional requirements for a computing machine (the mechanism). The machines are typically implemented by software programs operated on conventional digital computers. That this can be done at all is a demonstration of how well digital computers are useful for manifestation of abstractions.

While it is appropriate to think of a Miser as a machine with a variety of physical realizations, there is a mathematical theory that dictates the essential characteristics that each realization is expected to manifest. This is by design. We want Miser to be amenable to mathematical reasoning and analysis. The mathematical theory determines the correct behavior of Miser implementations. Even so, no result in the mathematical theory can ever be a proof about a Miser implementation. We want to develop an appreciation for how this is so and how there remains an useful connection between theories and concrete realizations even though the connection is a bridge that the theory can never cross.

The mathematical theory is essentially pure logic applied to a simple subject matter: the Miser Obs and functions over them. It is also the case that Miser procedures perform in logic-resembling ways and can achieve what is known as a computational logic. Both of these conditions require that we be careful about our demands on logic and how they will be expressed. We need to look at how the theory for Miser is expressed as a logical theory as well as how logic is expressed in Miser procedures (that is, by computational means).

This reliance on a logical theory will assist our differentiation among computation, logic, and mathematics. This will, in turn, help us clarify the relationship between procedures and functions and algorithms for functions as we look more closely at the Miser computational model.

The logical theory for oMiser, the foundation system, is symbolized Ot, short for Ob theory. It is an application of First-Order Logic.

1. First-Order Logic

The logical theory for Miser is mostly expressed in an informal style. Most of the assertions under the theory are in the form of equations or identities: equalities and inequalities.

Although there is not much need for a full-up expression in a formal logic, there is always an equivalent formal expression using First-Order Logic with equality (FOL=). The FOL= logical symbolism employs the following forms:

x = y equality: read as "x equals y"

x ≠ y inequality: read as "x not-equal-to y"

∀u(p) universal quantification, read as "for all u, p"

∃u(p) existential quantification, read as "there exists u such that p"

¬p negation: read as "not p", also seen as "~p" and "-p"

p ∧ q conjunction: read as "p and q", also seen as "p&q"

p ∨q disjunction: read as "p or q"

p → q conditional: read as "if p then q" or "p implies q", also seen as "p⇒q" or "p⊃q"

p ↔ q biconditional: read as "p if and only if q" or "p equivalent q", also seen as "x⇔y" or "x≡y"

A common order of precedence of the operations is indicated in the progression. The components of the equality and inequalities can be constants, variables, and expressions involving functions of operands that are the same kind of components. These are not applicative expressions and should not be read as Frugalese applicative operations (just yet, if ever).

A basic introduction to FOL notation is available on-line in the MIT Open Courseware [2]. There are other on-line descriptions [4, 6]. These should provide enough background to be able to read the formalisms of Ot.

The specific form used here is that described in the Handbook of Mathematical Logic [1]. Leisurely expositions with many examples and exercises are available in the books by Lemmon [3] and Suppes [5].

2. Informal Expression

The informal expression allows us to avoid heavy use of special symbols. For example, an axiom of equality,

∀x∀y∀z (x = y ∧ y = z → x = z)

could be expressed informally for Ot as

"For Obs x, y, z, if x = y and y = z, then x = z."

There are further examples to be found among the posts on Numbering Peano and in other Miser Project materials.

3. Ot as a Logical Theory

For Ot, there are additional constants to be introduced (e.g., ob_A, ob_B). We also presume a variety of functions (e.g., ob.a, ob.b, ob.ap). These and other additions to FOL= constitute Ot.

In particular, we assume that the variables of quantifiers (x and y in ∀x, ∃y) refer to Obs.

Because the domain of discourse, as it is known, is the Obs and only the Obs, how functions are represented in Ot becomes an interesting topic. The same is true for additional predicates (i.e., beyond the predicate implicitly associated with "="). We will explore these matters in further posts and articles of the Miser Project.

For now it is useful to point out that there is no direct way to express something about all/any functions or all/any predicates using FOL= (a consequence of being first-order). When we speak in such a way, it will have to be informally and outside of Ot. That situation is also to be explored further.

[update 2008-05-07T17:29Z: I couldn't resist changing the title from "The Logic of It" to "The Logic of Ot," a bad-pun opportunity which I hadn't noticed until making a diary entry about having completed the post. It is an apt phrasing, besides. As long as I am here, I also append a little more content to section 3.]

[1] Jon Barwise: An Introduction to First-Order Logic. Chapter A.1 (pp. 5-46) in Handbook of Mathematical Logic, Jon Barwise (ed). Studies in Logic and the Foundations of Mathematics, volume 90, Elsevier (Amsterdam: 1977). ISBN 0-444-86388-5 pbk.: The concerns I have expressed over Handbook do not apply to this chapter (although I just found a typo in the first example in section 3), which is tidy and makes useful connection to model-theoretic considerations. Our notational symbols are those used here.
[2] Leslie Kaelbling: First-Order Logic (lecture notes), Lecture 5, MIT Course 6.825 Techniques in Artificial Intelligence, Fall 2002. Open Courseware PDF file. accessed 2008-04-17.: These notes provide a leisurely and elementary introduction to FOL= notation with understandable applications to every-day logical situations.
[3] E. J. Lemmon: Beginning Logic. Van Nostrand Reinhold (Great Britain: 1965), modest revision edited by George W. D. Berry, Hackett Publishing (Indianapolis: 1978). ISBN 0-915144-50-6 pbk.: One advantage of this text, considered by its author to be more elementary than Suppes[5], with which it matches its notation, is that it is still used as a textbook and the author's insider remarks are entertaining. The Bibliography notes are recommended.
[4] PlanetMath: First-order logic. (article) accessed 2008-04-17.: This is a good place to find a summary of the notation without all of the subtleties and accompanying discourse. For a deeper look into how formal logic and logical theories work, Lemmon and Suppes will be rewarding [3, 5].
[5] Patrick Suppes: Introduction to Logic. Van Nostrand Reinhold (New York: 1957), Dover unabridged republication (New York: 1999). ISBN 0-486-40687-3 pbk.: Recommended for its gradual progression and build-up of first-order logic with useful examples and exercises. Chapter 11, on Functions, is a valuable bonus (as well as slightly contrary to what we have to say about functions in the context of the Miser Project).
[6] Wikipedia: First-order logic. (article) accessed 2008-04-17.

Miser: Frugalese for Applicative Operations

2008-02-29T19:44:00.001-08:00

Technorati Tags: models of computation, Miser, Frugal, Frugalese, applicative notation, functional programming

[update 2008-05-06T01:23Z: I corrected a small typo before linking to this post from another one in this series on Miser notions. I also included a reference to the work of Church where he introduces f(x, y) as equivalent to (f x) y.]

update 2008-04-12T17:49Z: I am still niggling with this post. There will be a separate post on some of these ideas, but I wanted to add something about being committed to treating functions as definite mathematical objects and viewing procedures as algorithms for (some of) them, even though we don't have a way to uniquely identify and distinguish functions (or procedures, for that matter). I think I'm content with this for now.

update 2008-04-12T02:45Z: After completing this post, I realized that all expressions that stand for functions are ones that characterize functions, but none of them "are" the function. I have revamped the post, today, to emphasize the introduction of functions by characterizing them. This makes some of the prose very awkward, especially if you just wanted to learn Frugal and use a Miser engine to produce something useful. It seems like a necessary foundational exercise for me, to ensure that I have successfully wrestled the relevant concepts to the ground. I took this occasion to do some tweaking to make some other points more clear, although adding the fuss about functions only being known by their characterization is probably a net loss. -- dh.]

On occasion, I mention that Miser is an applicative-programming system. There is a connection with the notion of functional programming. I want to clarify that terminology and, especially, illustrate the impact it has on notations for programming in applicative style.

The Basic Idea: Application as Fundamental
1.1 The logico-mathematical view of functions
1.2 The λ-calculus view of applicative functions
Be Ye Operand or Operator?
And in Frugalese?
Streamlining Applicative Notation
What About Multiple Parameters?
Left-Right, Left-Right, (Cha-cha-cha) Right-Left
Ah, Sugar, Sugar

1. The Basic Idea: Application as Fundamental

The key idea of applicative systems is that the fundamental operation is one of application of functions and their arguments or operands. This also involves the notion that functions are definite (mathematical) objects.

Consider, for example, the familiar notation f(x) from mathematics. In mathematics, this notation tends to be used

when the function is being used (that is, mentioned) in some larger expression and
when it is being described or characterized in terms of some essential relationships:

(1)    ( f(x₂) - f(x₁) ) / (x₂ - x₁)

(2)    f(i) = f(i-1) + f(i-2); i > 1

        f(1) = 1; f(0) = 0

illustrating the respective cases. In (2), f(n) is known as the n-th Fibonacci Number; we will allow ourselves to say that function f determines the Fibonacci Numbers.

1.1 The Logico-Mathematical View of Functions

In logic and mathematics, the mixture of usages to define/represent/characterize a function and to appeal to a function is commonplace. It is generally rare to consider f as identifying the function itself, as if functions have an independent existence, with f(x) as separately identifying an application of it.

In modern mathematics (i.e., that of the last 100 years or so), it has become useful to consider functions more directly. It happens

when defining the type of the function and
when characterizing the function in the form of an independent entity:

(1) f: N → N

(2) f = λi.(if i > 1 then f(i-1) + f(i-2) else 1)

(3) g = Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1)

The form (1) asserts that the function f has its domain of arguments in the natural numbers, N (according to the usual convention), and its range of results (→) is also in the natural numbers, N.

It is very easy to think of functions as depicting operations that produce something, thanks to the prevalence of computation in modern life, in teaching, and in our learning arithmetic (computation) before algebra and higher mathematics. Programming languages use "function" in their codes and their descriptions in a way that adds to this collapsing-together of functions, algorithms, procedures, and program code.

A more abstract way to think of individual functions is as specific relationships or correspondences of each distinct member of the domain with at most one member of the range. An element of the range may stand in the functional relationship to multiple members of the domain, but never vice versa. It is also possible that a function might not be defined at all, have no corresponding member of the range, for a given member of the domain.

Although this sense of functions as correspondences is perhaps more contemporary, our ordinary way of speaking is in terms of actions that determine results from given arguments. It is easy to ~~add~~ presume causality, speaking as if functions produce members of the range from members of the domain. The notations used in computer programming encourage this confusion of relationship and action or procedure. I notice that I tend to speak of the member of the range being determined by the function together with the selection of a member of the domain.

Computer folk can also think of this correspondence/relation characterization of functions as being realized by a database table with unique keys, each row having a field for the range value that is determined for the domain value represented by each key. This works in the case of finite domains and ranges, but we need something else when there is no such limitation (as is often the mathematical case).

In all of the forms that we've shown, the function is not being exhibited, it is being characterized. There are many characterizations of the same function and many different ways to characterize functions (as we are seeing here). There is no way to see or get your hands on "the function" because the concept is, in effect, abstracted from the relationship that it embodies (abstractly speaking). Nevertheless, we seem to be able to grasp what many functional relationships are from succinct characterizations of them. We rely on functional relationships in day-to-day use of mathematical entities (such as numbers) without having to think about all of this very much.

I'm bringing up this cloudy subject now because digging into the foundations of computing require that we have a sharper notion about functions and what it takes to trace the relationships from domain to range by computation.

1.2 The λ-calculus View of Applicative Functions

The form 1.1(2) involves a λ-expression (pronounced lambda-expression). This particular one, interpreted as the characterization of a function, happens to refer to the function that the expression is intended to characterize (singled out on the left) in the characterizing expression itself (on the right). The given characterization of f is recursive.

The form 1.1(3) specifies the same function in a way that has the characterization be completely accomplished in the right-hand side expression, with no appeal to the name of the function on the right-hand side of the equation. The special Y-operator is a function which transforms other functions into ones that operate recursively. In this case,

g(n) = (Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1)) n

        = if n > 1 then     (Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1))(n-1)
                                 + (Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1))(n-2)
                       else 1

illustrating that applications of functions having functions as operands can transform functions to other functions (as characterized).

The λ-calculus is important in Frugalese and for Miser. It is used here to emphasize that there are ways to characterize functions via self-contained λ-expressions. These λ-expressions can appear anywhere in the left or right of application operations, whether as functions, as in

(Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1)) n

or as operands, as

λfλi.(if i > 1 then f(i-1) + f(i-2) else 1)

is in

Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1).

The use of "=" with regard to formulae that characterize functions is a bit different than the "=" where we tend to think that a unique determination is happening. To remove any confusion between uniquelly-expressible mathematical entities (such as natural numbers) and the more-ellusive functions, we could have used "≡" or "∽" or some other indicator of equivalence. We won't be so fussy. So how do we determine that two characterizations that are not identical are still for the same function?

The usual approach is some version of the following procedure.

If it can be shown that f(x) = g(y) whenever x = y, then f = g, and vice versa. (We are begging some questions here, but that is the gist of it.)

2. Be Ye Operand or Operator?

What you shouldn't miss in the above, is this:

In applicative systems functions are treated as definite mathematical entities that are equally usable as operands (something a function can be applied to) and as functions (something that can be applied to an operand).

That is,

λfλi.(if i > 1 then f(i-1) + f(i-2) else 1) g

for some g: N → N

characterizes the same function as whatever

λi.(if i > 1 then g(i-1) + g(i-2) else 1)

does. And

λgλj.( g(g(j)) ) (Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1))

characterizes a function that is very nasty to compute (using the formula as a rule of computation) for increasing values of its natural-number operand.

By having functions be first-class candidates for use as operands and as functions, we have great freedom in characterizing functions, including in terms of application of functions in the establishment of other functions.

Although this seems pretty esoteric, it is useful to keep in mind that programming languages and compilers are literally embodiments of functions that derive (procedure codes for algorithms for) functions from their data (the source-program texts). There is a deep connection with some of the fundamental power of digital computers, and Miser is one way to explore that.

3. And in Frugalese ... ?

Frugalese is a language for expressing computations carried out by applicative systems such as Miser. Miser is a computational system, not a mathematical one. There is a mathematical connection, and that motivates our giving so much attention to the business of characterizing functions and what that does and does not accomplish.

For Miser and Frugalese, the fundamental operation is called "apply." Apply has two operands. Both operands are Obs. The first Ob is interpreted as a coding for the procedure to be followed. The second Ob is the data that the procedure will manipulate in some way.

We need to be clear, here, about what we mean by computation of a function. We mean that in the usual sense. I have here an operand, the representation of of a (definite) member of the function's domain, and I want to know the (definite) member of the function's range that the function determines, what we commonly refer to as "the result" of the function. An algorithm for a function, if we know one, is a (computational) procedure for arriving at the function-determined result given any valid operand. A coding of the procedure is an expression as data in a form that a computer can follow.

What puts the "functional programming" and "applicative-programming" into Miser and Frugalese is that one kind of computation of a function gives rise to the code for a procedure which can itself be used in the computation of another function, and so on. This doesn't change the computational idea, it just includes some very powerful and important cases at the heart of the power of digital computers.

Because of the connection between functions and procedures for their calculation, we re-use the mathematical notions as computational ones:

The Frugalese notation

p a

signifies an apply operation, where

p determines the Ob used as the first operand (the procedure code) and

a determines the Ob taken as the apply's second operand (the data to the procedure).

Apply operations are the Miser embodiment of the stored program principle:

Every Miser Ob can be used as either first or second operand of an apply.
Programs can treat data as programs and programs as data.
Programs can produce data results that works as programs.

The sense in which an Ob is being used as program or data is entirely determined by context. The same Ob may be used one way or the other at any instant.

Note that the apply operation is signified by simple juxtaposition of the term signifying the procedure and the term signifying the data.

What justifies our appropriating the mathematical notion of functional application as a notation for computation?

It is this connection.

If we can show that (Frugalese) (p a) = (q b) whenever a = b (a and b are the same Obs), then Obs p and q code (not necessarily the same) procedures for the same function: the same function is computed.

We cannot go so far as to say that p = q, because the codes can be different. Because we are here using "=" for identity of Obs, we would need a different symbol to signify that two obs taken as procedures are procedures for the same function. There is no such symbol in Frugalese, however. One motivation of the Miser Project is to sharpen our understanding of why there cannot be such a determination by computational means.

The vice versa case always applies in Miser and Frugalese, of course. If (as Obs), p = q, then (p a) = (p b) whenever (as Obs) a = b (and a result is determined by whatever p is, taken as code of a Miser procedure).

The Miser apply operation is also called ob.ap. It has been sketched a little in earlier posts.

From now on, we are illustrating Frugalese, not mathematical notation.

4. Streamlining Applicative Notation

Parentheses are used when it is necessary to control grouping into the operands for an apply operation:

p a

p(a)

(p) a

(p)(a)

all work.

In general,

p q r a

and p( q( r(a) ) )

express the same applicative operations. We will often use variations of the streamlined form, p q r a.

These are not equivalent to p q r a:

(p q r) a

(p q)(r a)

[Historical Note: Some applicative systems have p q r a be equivalent to ((p q) r) a. That is not the case for Frugalese. If you want that form, you will have to write it that way (or use the alternative form introduced below). If you run into one of these systems, you will have to be careful when transliterating between that notation, Frugalese, and vice versa.]

5. What About Multiple Parameters?

In Frugalese there are no additional operands in a single apply operation. In Miser terms, there's one Ob for the procedure and one Ob for the data. That's it. One way to introduce multiple-operands of functions is by multiple apply operations:

p(q, r, a) is equivalent to ((p q) r) a

keeping in mind that however p is constructed, it has to be such that we end up with what we have in mind for p(q, r, a).

In general,

f(x₁, x₂, ..., x_n)

= f(x₁) (x₂, ..., x_n)

= f(x₁, x₂) (..., x_n)

= f(x₁, x₂, ...) (x_n)

= (... (f x₁) x₂) ...) x_n.

This particular device was introduced by Alonzo Church, the inventor of the λ-calculus [1]. The basic idea is as follows.

In an applicative expression such as

plus(1, 2)

the 1 and 2 are not two operands of the function plus, the applicative interpretation is

plus(1) 2

where plus(1) is a new procedure that is then applied to to 2 to determine the result.

If we're very serious about wanting to have a procedure that works on several elements of data at once, rather than deliver intermediate procedures, we can bundle several operand values into a list which is then used as a single data operand in the apply operation:

sum[1, 2, 3]

has the single operand [1, 2, 3], which is a list of the individual operand values 1, 2, and 3.

The choice between an applicative evaluation such as

plus(plus(1, 2), 3)

and sum[1, 2, 3]

will depend on the context and whether it is important to single out plus(plus(1,2)), say, as an important intermediate function. (This is a trivial case meant to illustrate a broader useful practice.)

6. Left-Right, Left-Right, (Cha-cha-cha) Right-Left

In the previous section, there is an additional shorthand that needs to be explained.

p q r a

is p( q( r(a) ) )

but p q(r) a

is p((q r) a) or p q(r, a)

and not p( q( (r)(a))).

The rule is that bracketed elements are data of the procedure immediately to the left, if any. This is worked out in left-to-right order. For example,

s p(q)(r) a

= s( ((p q) r) a )

= s p(q, r, a)

= s p(q, r) a

among others. The choice of particular form depends on the circumstances, the need for parentheses to control essential grouping, and how one might want the features of the applicative expression to stand out.

This rule also applies for list-shaped operands:

s p[x,y](r) a

= s( ((p[x,y]) r) a )

= s p([x,y], r) a

7. Ah, Sugar, Sugar

For Miser, all of these variations are simply syntactic sugar, a sweetening of expression sequence and bracketed groupings that always boils down to a particular combination of apply operations. The many different ways to group in applicative expressions is a way to express some sense of what the purpose of the expression is. At some point, there are variants of the notation that I find myself using quite naturally and in preference to conventional functional notation.

The grouping rules are straightforward but it is easy to make mistakes with them. Small differences can have a big impact. For example, it is inappropriate to rewrite

s p(q, r) a

as s (p)(q, r) a

which is the same as s(p, q, r) a instead of s(p(q, r) a).

So long as everything is an Ob in the use of Frugalese notation for Miser, there is very little to help detect unintended combinations. The beauty of Miser is that everything is an Ob and Obs are interchangeable as data and as procedure (codes). The difficulty of Miser is that there is no such thing as an inadmissible apply combination.

This also means that there is no perfect way to pretty-print any Miser applicative expression, although using a pretty-printer transformation into some sort of standard form might reveal an unintended consequence. That's something worth looking at.

There are more notational features in Frugalese. The focus here has been on the different ways that apply operations can be expressed for carrying out in a Miser implementation. The purpose, beside taking a try at explaining it, is to equip ourselves to start putting the notation to good use.

[1] Alonzo Church: An Unsolvable Problem of Elementary Number Theory. American Journal of Mathematics 58 (1936), 345-363. Reprinted in pp. 88-107 of The Undecidable: Basic Papers on Undecidable Propositions, Unsolvable Problems and Computable Functions, Martin Davis (ed.), Raven Press (New York: 1965), ISBN 0-911216-01-4.: Here Church introduces λ-expressions as part of exploring a bigger question with regard to the unavailability of effective procedures for an infinitely large set of problems. The original formulation of Church's Thesis is a footnote in this paper.
Church's tendency was to bracket formulae for functions in {...} and for operands in (...), stating: "A formula {F}(X) may be abbreviated as F(X) in any case where F is or is represented by a single symbol. A formula {{F}(X)}(Y) may be abbreviated as {F}(X, Y), or, if F is represented by a single symbol, as F(X, Y). And ... ." Here Church is careful to use F, X, Y, etc., as variables for formulas that may be substituted in the given forms.

SeaFunc: 2008-02-20 Functional Programming Meetup

2008-02-16T09:13:00.001-08:00

Technorati Tags: SeaFunc, Seattle, Northwest, Functional Programming, Lisp, F#, Haskell, OCaml, Scheme

Who: SeaFunc, the Seattle area Functional Programming interest meet-up

When: 8:00 pm, Wednesday, February 20

Where: Ralph's Grocery Deli
2035 4th Avenue
Seattle, WA 98121

We will be meeting in the cafe area against the South wall. The Seattle web logger meetup will also be congregating at Ralph's, starting at 7:00 pm. Yes, there is free WiFi.

These are informal gatherings with the discussion following whatever topics the attendees are interested in and fascinated by at the moment.

Anyone interested in the application, extension, or construction of advanced programming languages, functional or not, is encouraged to come.

Although there is a shared interest in functional-programming systems and their languages (Haskell, Scheme, Lisp, ML, OCaml, Scala, F#, ..., <your favorite here>), the attendees also share interest in general advanced programming-language concepts. Many of these have historical ties to functional programming:

Interactive development systems and dynamic debuggers
DSLs: Domain Specific Languages
Declarative Programming
Dynamic typing and Dynamic scripting languages
Generic programming
Garbage-collection systems and automatic management of dynamic objects
Parallel programming
Logic programming
Program verification and correctness checking systems
Model-based program development and maintenance

SeaFunc began informal meetings in June 2004. There is a low-volume Yahoo! Group used primarily for meeting notices and notices about related groups that form from time to time (such as LispSEA, a Lisp-programming interest and advocacy group), interspersed with occasional technical questions and other announcements. There are no dues or other formalities. It's all volunteer-driven and self-organizing.

Recently, SeaFunc began alternating meetings in the central Seattle area and the East Side (Bellevue-Redmond) area. The February 20 meeting is the ~~third~~ fourth gathering of 2008.

Also of Interest:

The initial meeting of a Northwest Functional Programming Interest Group is being held at the Seattle Central Library, 1000 4th Avenue, Seattle, from 5:00 - 6:00 pm, also on February 20. This get-together is an opportunity to meet others interested in the subject and also ponder the prospects for meetings based around presentations and guest speakers. A functional-programming interest double-header!

For further information, contact L. G. Meredith at Biosimilarity LLC, +1 206.650.3740.

I'll be at both events to see how we might achieve critical mass together.

Miser: The Immutability of Obs

2008-01-28T22:34:00.001-08:00

Technorati Tags: PLT Scheme, immutable objects, models of computation, distributed computation, manifest mathematical objects, Miser, oMiser

In the Miser formulation of a functional/applicative computing machine, the fundamental data structures, the obs, are immutable. There is no operation that can change an ob. Obs behave very much as mathematical objects (as opposed to the object-oriented programming sort), and mathematical reasoning is applicable to obs and the operations defined on obs. This is a very strict condition. It is intended that the computer manifestation of obs be as indistinguishable from the mathematical as possible given the realities of computational speed and memory-capacity limitations.

There is no way to alter an ob. It is as if obs are neither created nor destroyed, they are merely found. It takes some work to maintain such an illusion, especially in a computation distributed across multiple computers, but it is critical to the formulation of Miser that it be made so.

This immutability also prevents the appearance of cyclic ob structures except for the special cases of singletons and only singletons being immutably self-cyclic [2]. Immutability also allows a single, well-defined equality relation, signified in the usual way by "=".

Immutability First, Mutability Another Way

The purpose has been to start with a "pure" system in this way. When mutability is finally entertained, it will be in ways that are strictly-confined and that do not alter the immutability of obs in any way whatsoever.

This can be done with Miser simply by breaking its connection with those functional-programming systems, especially Lisp and Scheme, that have allowed relatively unrestricted mutability of their fundamental data structure from the beginning. I don't know if it was a mistake to have done that, although I long thought so. I do know that it would be a mistake for that to be done with Miser. Although later functional-programming systems have been more generous with immutability, I want to take a fresh look that begins with immutability as a given.

Snatching from the Jaws of Mutability

Meanwhile, serious, practical interest has arisen in having immutable data structures where that was not the natural state of affairs before.

Really un-mutable Scheme. It was amusing to see that version 4.0 of PLT Scheme is going to have the fundamental pairing operation for list structures (the Lisp cons[x; y]) involve immutable structures [1]. This is part of a shift from mutability as the default to mutability as the exception. There will be mutable forms of the data structures, but new operations are required for using them. In a comment, it is revealed that the opportunity to rely on a single "=" is not being taken just yet.

There is also interest in operating with immutable structures in object-oriented systems. Eric Lippert provides a key element to the basis for such interest [3]:

"Objects which are truly madly deeply immutable have a lot of great properties. They are 100% thread safe, for example, since obviously there will be no conflicts between readers and (non-existent) writers. They are easier to reason about than objects which can change. But their strict requirements may be more than we need, or more than is practical to achieve."

Lippert is exploring ways to have deeply immutable objects when they are really useful, but not having that be fundamental or always be required. His programming examples that are used to tease out deep immutability for practical situations are very challenging and useful to explore. A systematic approach and further examples can be found in the work of Chris Okasaki [4, 6].

Although thread safety is an important benefit, there are others:

Immutable data structures are inherently distributable, whether completely or partially, and they can be replicated without concern for synchronization since they never change.
Partitioning of a problem and introduction of parallel processing can sometimes be extremely efficient with quasi-immutable data structures [5].

Interest in immutability seems to be arriving hand-in-hand with awakening interest in functional programming as well.

The inherent distributability and thread safety, as well as the amenability to mathematical treatment, are the motivations for persisting in immutability-first for Miser.

[1] Matthew Flatt: Getting rid of set-car! and set-cdr! PLT Scheme (web log), 2007-11-12.
[2] Dennis E. Hamilton: Miser Hacks II: A Hole to Bind Them. Numbering Peano (web log), 2007-11-04.
[3] Eric Lippert: Immutability in C# Part 10: A double-ended queue. Fabulous Adventures in Coding (web log), msdn.com, 2008-01-22.The ongoing series is under the immutability category. The initial posting provides a nice classification of immutability du jour, and an earlier series develops some interesting data structures for an A* search algorithm.
[4] Chris Okasaki: Purely Functional Data Structures. Cambridge University Press (Cambridge: 1998), ISBN 0-521-66350-4 pbk.: An earlier version of the material, with some differences in coverage, can be found in Okasaki's 1996 thesis.
[5] Jeffrey Dean and Sanjay Ghemawat: MapReduce: Simplified Data Processing on Large Clusters. Comm. ACM 51, 1 (January 2008), 107-113.: The original 2004 Google Research publication is available on-line.
[6] Chris Okasaki: Ten Years of Purely Functional Data Structures. Teaching, Playing, and Programming (web log), 2008-02-08 (via Lambda the Ultimate).: A wonderful retrospective on functional data structures, the anguish of immutability, and how it is overcome. It will be interesting to see how some of the remedies fail or survive under distribution of the data structures.

[update 2008-02-08T12:13 -0800: I was bowled over to see that Chris Okasaki is blogging and now his reflections on Purely Functional Data Structures have been posted in a wonderfully literate essay. That is reflected in the update here. I finally remembered that the C.ACM version of [5] is available on-line and I now link to it here. I finally realized that my use of relative links to fragments in these posts is problematic on the front pages of my blogs. I am not going to do anything about that. It's a great example of failing to consider the overall system, though.]

Computer Science Classics: Your Picks?

2008-01-17T17:47:00.001-08:00

Technorati Tags: Computer Science, ACM, Computer Science Classics, ACM Downloads

Some time ago, I participated in an on-line discussion and poll for identifying those out-of-print Computer Science Classics that are the favorites of ACM members. The polling was distilled down to the 25-book ACM Classic Books Series with arrangements to make the books again available.

Thanks to Paul McJones, I see that twelve classics are now available as PDF downloads. If you do not have an ACM Subscriber membership, you can register to receive limited free service by following the link at the top of the Classics page. That is all you need to download the Classics PDFs.

McJones lists all of the PDFs that are available but for Aho and Ullman's Theory of Parsing, Translation, and Compiling. Ehud Lamm mentions four of likely interest to programming-language enthusiasts (also omitting Aho and Ullman, having me wonder if this is a more-recent addition to PDFs).

My Selections

Here are the ones that I downloaded or have alternatives to:

E. F. Codd. The Relational Model for Database Management: Version 2 (1990). Not downloaded. I had this book at one time. Currently, I find the original foundation papers and the analysis by Chris Date to be more useful.
C. A. R. Hoare and C. B. Jones (editor). Essays in Computing Science (1989). Here are 22 essays starting with Sir Tony's Turing Award Lecture, "The Emperor's Old Clothes," and including his analysis of the Quicksort algorithm, the unfortunately never-adopted "Contribution to the Development of ALGOL" (with Niklaus Wirth), a complete bibliography (up to the time of publication), and much more. It's great to have so many of these in one place, those I recall and those I must remedy my neglect of.
Dorothy Elizabeth Robling Denning. Cryptography and Data Security (1982). A basic reference that I missed as I delved into computer systems security in my M.Sc in IT studies.
Seymour Papert. Mindstorms: Children, Computers, and Powerful Ideas (1980). Not anything that I had read before. I am curious about for Papert's influence on the One Laptop Per Child program and its initial XO-1 computer system and it seems useful to review this work.
Alfred V. Aho and Jeffrey D. Ullman. The Theory of Parsing, Translation, and Compiling (1972). I've never owned this book, but it is a fundamental reference and now I have it.
O.-J. Dahl, E. W. Dijkstra, and C. A. R. Hoare. Structured Programming (1972).
John F. Hopcroft and Jeffrey D. Ullman. Formal Languages and Their Relation to Automata (1969) was not downloaded, because I have the 2001 edition of Introduction to Automata Theory, Languages, and Computation. I may be mistaken in the adequacy of the latter's coverage of the latest thinking on material in the former, but I can wait to find that out. I also think this is a book that I had once before and let slip away.
IBM. IBM System/360 Principles of Operation (1964). An amazing achievement at the time, there is much to be learned from the Principles of Operation approach to many current elements of computing. Something for review and taking lessons from.

On reviewing the list just now, I retrieved one other:

Apple. Macintosh Human Interface Guidelines (1992).

What I Yearn For

And if there were PDFs available, I would have added these:

Jon Louis Bentley. Writing Efficient Programs (1982)
Edsger W. Dijkstra. Selected Writings on Computing: A Personal Perspective (1982)
Edward Nash Yourdon (ed.). Classics in Software Engineering (1979). I have no idea why I let this book get away from me.
John Allen. Anatomy of LISP (1978). See previous comment.
Kathleen Jensen, Niklaus Wirth, et.al. PASCAL User Manual and Report (1974). Well, I don't miss my copy that much, but I bet there's more to learn by having it to refer back to now.
Kenneth E. Iverson. A Programming Language (1962). Along with the John Allen book, I miss this one the most.

OK, What Are Your Picks?

It will be interesting to see what the download statistics will be for these books. Meanwhile, what are your picks?

[update 2008-01-18T20:20Z I added a missing word and took the opportunity to clean up a few of my lengthy sentences.]

Cybersmith: How Many Functions Am I Holding Up?

2008-01-07T14:20:00.001-08:00

Technorati Tags: programming idioms, programming style, functional programming, object-oriented styles, oMiser, cybersmith

This post is intended to accomplish three things:

Find a decent way to have Windows Live Writer produce pretty-printed code that appears cleanly on both blog pages and in my RSS feeds.
Refine my definition of a function that has a critical role in the Miser Project.
Object to a strange set of commandments about the size and other characteristics of "functions," exploring how to make them work in functional programming while recognizing the compromises involved.

How Big Should a Function Be? When I first saw the commandment to describe computational functions in around 5 lines, I thought it was a joke. With little consideration for the language or the situation, I found the injunction about keeping function descriptions small (as opposed to functional) simply bizarre. Yet it is from a very smart fellow [1], and I was stopped in my tracks. Although Martin concedes that this is situational, there is no guidance about that. I worry that budding cybersmiths will take this as gospel and a commandment not to be broken.

The echo chamber in the comments, and the dismissal of those who objected to context-free generalizations is surprising. That supports my concern. There's too much fussiness over indentation, nesting of conditionals and other layout considerations. This seemed to flaunt the principled sophistication of guidelines that I favor, such as the C++ Coding Standards of Sutter and Alexandrescu [2].

Breaking the Rules on Principle

Here's the test case that came to mind as soon as I read Martin's article.

The programming notation is Frugalese, an informal, unimplemented language that I use to illustrate functional-programming concepts. That's unimportant. It is important that it is neither Java nor C# and it is completely functional. Frugalese is an applicative notation for computation: the fundamental operation consists of carrying out a procedure described by one operand against the data represented by the second.

The operation ob.ap[f, x] defines the basic interpretive process of oMiser[3]. The operand f is taken as a program and the operand x is taken as data which the program is to process. In oMiser these are both data structures called obs. The Frugalese example is an applicative expression of the oMiser interpreter's basic applicative operation. (That is, we are using an applicative-language notation to describe the implementation of an applicative-machine interpreter. There's no harm in that and there is no chicken-and-egg problem: the oMiser implementation used by oFrugal is not implemented in oFrugal or any higher-level Frugalese language. But ob.ap, below, is a faithful simulation, capturing the essential behavior and functionality.)

I choose this example because I am in the process of rethinking its formulation and how to present it in a direct way. It struck me that this definition illustrates the difficulties and trade-offs of arbitrarily choosing smaller function (procedure) definitions over larger ones.

In this example, there is only the one function, ob.ap, that is defined for use from elsewhere. There are 23 lines, not counting blank lines introduced for layout purposes:


  1:  defrec         /* SELF-CONTAINED SINGLE FUNCTION DEFINITION */
  2: 
  3:     ob.ap[f, x]         /* Apply the procedure defined by f to operand x */
  4:
  5:        =  if    is-individual(f)  /* Atomic (single-instruction) operation */
  6:
  7:           then  if    f = ob_A  then  ob.a x      /* a-part of x */
  8:
  9:                  elif f = ob_B  then  ob.b x      /* b-part of x */
 10:
 11:                  elif f = ob_E  then  ob.e x        /* enquote x */
 12:
 13:                  else ob.c(f, x)           /* prefix f to x, etc. */
 14:
 15:           elif  is-singleton(f) then  ob.a f      /* quoted constant */
 16: 
 17:           else  eval f                     /* Evaluate applicative form f */
 18:
 19:                  where rec eval p  
 20: 
 21:                          =  if    is-singleton(p)   /* argument reference */
 22:
 23:                             then if      p = ob_ARG     then  x
 24:
 25:                                   elif   p = ob_SELF    then  f
 26:
 27:                                   else   ob.a p   /* or (quoted) literal */
 28:
 29:                             else  let   op = ob.a p,
 30:
 31:                                         arg = ob.b p
 32:
 33:                                    in if   op = ob_C     /* pairing form */
 34:                                       then ob.c( eval ob.a arg,
 35:                                                   eval ob.b arg )
 36:
 37:                                       elif op = ob_D  /* comparison form */
 38:                                       then ob.d( eval ob.a arg,
 39:                                                   eval ob.b arg )
 40:                                                       /* applicative form */
 41:                                       else ob.ap[ eval op, eval arg ];

The Principle Function

There are at least two functions defined above. The main one is ob.ap[f, x]. This definition is self-referential (signified by defrec), where the defined function is referenced within its definition at line 41. The operation of evaluating this function can lead to multiple recursive evaluations using the same function. The essence of the function is this simple structure:


  1:  defrec
  2: 
  3:     ob.ap[f, x]         /* Apply the procedure defined by f to operand x */
  4:
  5:        =  if    is-individual(f)  
  6:
  7:           then  /* do the simple cases for f being simple */
 

 17:           else  eval f  /* handle the case of f being a composed form */

But Wait, There's More

The function for the composed-form evaluation, eval(f), is defined in the following lines. Here's the key definition:


  1:  defrec
  2: 
  3:     ob.ap[f, x]          /* Apply the procedure defined by f to operand x */
 
 ...
  







 19:                  where rec eval p          /* Evaluate applicative form p */
 20: 
 21:                          =  if    is-singleton(p)   /* argument reference */
 22:
 23:                             then if      p = ob_ARG     then  x
 24:
 25:                                   elif   p = ob_SELF    then  f
 26:
 27:                                   else   ob.a p   /* or (quoted) literal */
 28:
 29:                             else  /* cases for composite p */
 
 41:                                       ... ob.ap[ eval op, eval arg ];

This function is being defined local to the fragment



 

 17:           else  eval f  /* handle the case of f being a composed form */

That is, the function is only usable (and only defined for use) in conjunction with the expression eval f.

The where introduces a local definition and it introduces it right there. This establishes that this is the only place that definition is required. This makes the dependency exactly clear in both directions: ob.ap(f, x) only requires eval in that one place and eval is only used in one way (other than the ways that eval is recursive).

Equally important is the fact that eval depends on the parameters of ob.ap[f, x]. This is at lines 23 and 25. This is another isolated dependency. It is global to eval yet local to the instance of ob.ap[f, x] in which line 17 is reached. In effect, the definition of eval(p) occurs dynamically in each instance of ob.ap[f, x] evaluation. That instance-specific definition is reused for all recursions of the defined eval.

These kinds of "inner function" have been around since the introductions of the LISP and ALGOL 60 programming languages.

This use of the inner function, a recursive one at that, is very economical. It may not the clearest, but it emphasizes locality and cohesion that are important for this particular algorithm.

Breaking Up Is Maybe Not So Hard To Do?

It is possible to break out the subfunctions used in defining ob.ap[f, x]. These considerations must be weighed:

The contracted interface and definition is that of ob.ap[f, x]. Everything else is incidental and the definition can be replaced by anything that preserves the required behavior. In one sense, this makes refactoring of the interior of the definition permissible. But,
The exposure of additional interfaces creates a separation of concerns that is not required by the definition of ob.ap[f, x]. When a subfunction is no longer local, there is the problem of the maintenance of the dependency with ob.ap[f, x].
The extraction of a subfunction requires expansion of variability beyond that required by locally-incorporated subfunctions. This makes the extracted subfunction more complex than the embedded one.

To see what's involved, pull the definition of eval p outside of the definition of ob.ap[f, x]:


  1:  defrec
  2: 
  3:     ob.ap[f, x]         /* Apply the procedure defined by f to operand x */
  4:
  5:        =  if    is-individual(f)  /* Atomic (single-instruction) operation */
  6:
  7:           then  if    f = ob_A  then  ob.a x      /* a-part of x */
  8:
  9:                  elif f = ob_B  then  ob.b x      /* b-part of x */
 10:
 11:                  elif f = ob_E  then  ob.e x        /* enquote x */
 12:
 13:                  else ob.c(f, x)           /* prefix f to x, etc. */
 14:
 15:           elif  is-singleton(f) then  ob.a f      /* quoted constant */
 16: 
 17:           else  ev[f,x] f,                  /* Evaluate applicative form f */
 18:
 19:     ev[f,x] p  
 20: 
 21:        =  if    is-singleton(p)   /* argument reference */
 22:
 23:           then if      p = ob_ARG     then  x
 24:
 25:                 elif   p = ob_SELF    then  f
 26:
 27:                 else   ob.a p   /* or (quoted) literal */
 28:
 29:           else  ... /* UH OH */
 30:

We have succeeded in pulling out ev[f, x] p in place of the internal eval p definition.

The additional parameters, [f, x], are required because we are no longer in any context where those variables are visible globally. In effect, it is necessary to replace eval everywhere in the definition of eval p with ev[f, x].

I don't want to write a redundant ev[f, x] for every eval in the original version. There is not meant to be any variability here, so I choose another approach. This involves defining a third function. There are many other ways to introduce that third function. My choice takes advantage of the fact that a let ... in form is equivalent to introduction of a function definition and application:


  1:  defrec     /* INTER-DEPENDENT MULTIPLE-FUNCTION DEFINITION */
  2: 
  3:     ob.ap[f, x]         /* Apply the procedure defined by f to operand x */
  4:
  5:        =  if    is-individual(f)  /* Atomic (single-instruction) operation */
  6:
  7:           then  if    f = ob_A  then  ob.a x      /* a-part of x */
  8:
  9:                  elif f = ob_B  then  ob.b x      /* b-part of x */
 10:
 11:                  elif f = ob_E  then  ob.e x        /* enquote x */
 12:
 13:                  else ob.c(f, x)           /* prefix f to x, etc. */
 14:
 15:           elif  is-singleton(f) then  ob.a f      /* quoted constant */
 16: 
 17:           else  ev[f,x] f,                  /* Evaluate applicative form f */
 18:
 19:     ev[f,x] p  
 20: 
 21:        =  if    is-singleton(p)   /* argument reference */
 22:
 23:           then if      p = ob_ARG     then  x
 24:
 25:                 elif   p = ob_SELF    then  f
 26:
 27:                 else   ob.a p   /* or (quoted) literal */
 28:
 29:           else evform(ev[f,x], ob.a p, ob.b p),
 30:
 31:     evform(eval, op, arg)
 32:
 33:        =  if   op = ob_C     /* pairing form */
 34:           then ob.c( eval ob.a arg,
 35:                       eval ob.b arg )
 36:
 37:           elif op = ob_D  /* comparison form */
 38:           then ob.d( eval ob.a arg,
 39:                       eval ob.b arg )
 40:                           /* applicative form */
 41:           else ob.ap[ eval op, eval arg ];

This refactoring works easily because ev[f, x] is a legitimate intermediate result that can be defined and used as an individual operand.

The use of evform(ev[f, x], ob.a p, ob.b p) in the definition of ev[f, x] p is very much like saying


 29:           else let eval = ev[f,x],
 30:                       op = ob.a p,
 31:                      arg = ob.b p
 32:
 33:                  in if   op = ob_C     /* pairing form */
 34:                     then ob.c( eval ob.a arg,
 35:                                 eval ob.b arg )
 36:
 37:                     elif op = ob_D  /* comparison form */
 38:                     then ob.d( eval ob.a arg,
 39:                                 eval ob.b arg )
 40:                                     /* applicative form */
 41:                     else ob.ap[ eval op, eval arg ];

This second form preserves locality at the cost of additional indentation and coupling directly into the superior function. At the same time, it avoids surfacing a function that is only needed in one place.

Confessions and Compromise

It is intriguing that the form with definitions of three functions satisfies Martin's requirements[1].

Confession: When I explain the operation of ob.ap[f, x], I do so using auxiliary function definitions[3]. I use more of them than the three factored out here. One important reason for additional functions is the prospect that some of them are extended as more features are added to the system.

Compromise: When I want to control the coupling among functions, it is important to have an explicit way to reflect that. Locality of auxiliary function definitions allows that. My compromise version would preserve the three functions for explanatory purposes in this locality-controlling form:


  1:  defrec    /* SELF-CONTAINED WITH AUXILARY FUNCTION DEFINITIONS */
  2: 
  3:     ob.ap[f, x]         /* Apply the procedure defined by f to operand x */
  4:
  5:     =  if    is-individual(f)  /* Atomic (single-instruction) operation */
  6:
  7:        then  if    f = ob_A  then  ob.a x      /* a-part of x */
  8:
  9:               elif f = ob_B  then  ob.b x      /* b-part of x */
 10:
 11:               elif f = ob_E  then  ob.e x        /* enquote x */
 12:
 13:               else ob.c(f, x)           /* prefix f to x, etc. */
 14:
 15:        elif  is-singleton(f) then  ob.a f      /* quoted constant */
 16: 
 17:        else  ev[f,x] f,                  /* Evaluate applicative form f */
 18:
 19:     where ev[f,x] p  
 20: 
 21:     =  if    is-singleton(p)   /* argument reference */
 22:
 23:        then if      p = ob_ARG     then  x
 24:
 25:              elif   p = ob_SELF    then  f
 26:
 27:              else   ob.a p   /* or (quoted) literal */
 28:
 29:        else evform(ev[f,x], ob.a p, ob.b p),
 30:
 31:     where evform(eval, op, arg)
 32:
 33:     =  if   op = ob_C     /* pairing form */
 34:        then ob.c( eval ob.a arg,
 35:                    eval ob.b arg )
 36:
 37:        elif op = ob_D  /* comparison form */
 38:        then ob.d( eval ob.a arg,
 39:                    eval ob.b arg )
 40:                        /* applicative form */
 41:        else ob.ap[ eval op, eval arg ];

The use of where at the "top level" is a bit like saying private in the method declarations for a class. But we do not need to introduce classes, saving us from having to consider the interactions between access specifiers and inheritance.

More confession: I just made up this additional use of where as an "access" specifier similar to use of rec to impose self-reference in definitions. I invented it on the spot. (The internal form of where clause has been known at least since the work of Burge and Landin[4, 5] and was used by Strachey) I introduced this specifier flavor because I am jealous of the improved readability of the interdependent multi-function definition form.

I will pay a penalty in the complication of the Frugalese grammar and in explanation of how this use of where impacts the definitions in a definition list where it occurs.

This device is not going to assure that Martin's injunction can or should be satisfied in all cases[1]. I'm pleased that I can take advantage of that style without surrendering localization of interdependencies among function definitions. I'm going to use this Frugalese feature much more before I am satisfied that it carries the day well enough.

In testing Martin's guidelines against a specific example of my own, I found a way where following the rules slavishly would surrender other qualities of functional definitions. Since I have the privilege of defining the Frugalese notation, I am able to adopt a compromise approach that keeps auxiliary functions auxiliary while flattening the layout and the presentation. It remains to be determined whether this addition is a significant improvement overall.

[1] Robert C. Martin: How Big Should a Function Be? Uncle Bob's Blatherings (web log), Object Mentor, 2007-10-29 (via Hans Martin Kern).: "Functions should be small. Very small. A well written function might be 4, 5, perhaps 8 lines long; but no longer. ... Specifically, functions that process information as part of an algorithm should be very small." The ob.ap[f, x] definition is definitely that of an algorithm that processes information, so it is not at all clear to me how that fact correlates with having to be small, to have a brief description. Even though I manage to break out the subfunctions, I could not be satisfied without also enforcing their auxiliary nature.
   "High level functions should be followed by lower level functions that should be followed by still lower level functions. The calls should all point downwards in the source file if possible." This is certainly satisfied in the definition of ob.ap[f, x] in all of the variations shown.
   "Of course these rules cannot be followed 100% of the time, but they are good rules nonetheless. Our goal, as programmers, is to write programs that humans can understand and maintain." is a reasonable statement, but for its sequel that seems to suggest there is an obvious boundary between "good" and "bad": "Remember, a product that works, but that has a bad internal structure is a bad product."   I suppose my view can be summarized as a willingness to support such practices along with a reluctance to enforce them.
[2] Herb Sutter and Andrei Alexandrescu: C++ Coding Standards -- 101 Rules, Guidelines, and Best Practices. Addison-Wesley (Boston: 2005), ISBN 0-321-11358-6 pbk.: The principle that struck me in this context was "0. Don't sweat the small stuff." Some others that apply to this example are "5. Give one entity one cohesive responsibility," "10. Minimize global and shared data," "18. Declare variables as locally as possible," and "20. Avoid long functions. Avoid deep nesting" (with statement of the principle involved). You can decide how well "6. Correctness, simplicity, and clarity come first."
[3] Dennis E. Hamilton: oMiser Sketch [latest], Miser Theory Notes page N020600f 0.05 2007-05-03.: I am in the process of shrinking and correcting much of this material, my motivation for finding a succinct statement of ob.ap(f, x).
[4] William H. Burge: Recursive Programming Techniques. Addison-Wesley (Reading, MA: 1975). ISBN 0-201-14450-8.: I rely on this approach to functional programming for many idioms and test cases for Frugalese. The structure of Frugalese is different from ISWIM in several respects, although it is generally easy to translate between the notations.
[5] Peter J. Landin: The Next 700 Programming Languages. Comm. ACM 9, 3 (March 1966), pp. 157-164.: The refactoring performed between my variations of ob.ap[f, x] definition is based on a few of the rewriting cases described by Peter Landin in this paper. Although I also use indentation heavily when writing Frugalese, that is not essential to the language (so far).

More Annotated Turing

2007-11-22T21:45:00.001-08:00

Technorati Tags: Charles Petzold, Alan Turing, Turing Machine, Universal Turing Machine, UTM, Church-Turing Thesis, Computation Theory, On Computable Numbers, orcmid

A Book's Progress

Before digital computers could do much of anything,
Alan Turing demonstrated what they could never do....

I fancy the tag line that Charles Petzold has placed on his new page about his forthcoming book, The Annotated Turing.

It is now possible to pre-order the book. I placed my order on amazon.com (US). I think the provisional cover illustration is marvelous and I'm holding onto this copy of the image just in case they decide to change it (but please don't).

I'm eager to have the book and follow Petzold's narrative through Turing's paper.

The Mathematical-Logic Connection

For those who want more on the context of this activity, there are some other useful books on the mathematical-logic and computability aspects of Turing's work. Here are some selections:

Berlinski, David.: The Advent of the Algorithm: The Idea that Rules the World. Harcourt (New York: 2000), ISBN 0-15-100338-6. A poetic excursion into Liebniz's dream and the quest that led to the digital computer and more.; Davis, Martin.; Engines of Logic: Mathematicians and the Origin of the Computer. W.W. Norton (New York: 2000), ISBN 0-393-32229-7 pbk. An account of the contributors from Liebniz to Turing, to the modern digital computer and the puzzling questions raised around who we are and how we operate as we do. There is a careful non-mathematical illustration of Turing Machines and some of the key ideas associated with them.; Davis, Martin (ed.); The Undecidable: Basic Papers on Undecidable Propositions, Unsolvable Problems and Computable Functions. Raven Press (New York: 1965), ISBN 0-911216-01-4. A valuable collection of fundamental papers including those of Gödel, Church, and Turing. The Petzold book will be an useful introduction to this kind of mathematical approach and the related work. The Turing paper is here.

There are texts on Computation and Computability theory. The Wikipedia treatments provide a general sense of the different focuses that have followed from the work of Turing and others.

Turing the Person

About Alan Mathison Turing, there is the work of his biographer, Andrew Hodges, as well as the Turing Digital Archive, where Turing's publications can be found along with unpublished materials and memorabilia. The Turing paper that is the subject of Petzold's book can also be examined on-line there.

Different (Universal) Computation Models

2007-11-17T15:47:00.001-08:00

Technorati Tags: Stephen Wolfram, Alex Smith, Foundations of Mathematics, FOM, Universal Turing Machine, Alternative Computation Models, Miser Project

On October 24, 2007, Stephen Wolfram announced the awarding of a $250,000 prize for a proof that a particular 2-state 3-color Turing Machine is indeed universal. [I selected a specific Wikipedia page on Wolfram because it covers the prize and the ensuing discussion on FOM in a form that is accurate enough at this point. It is unimportant to this post how later developments may be reported there.]

update 2007-11-22: The challenging and testing of the (2,3)-TM's universality is continuing. I have added some additional links and I am now going to take my attention off of this activity. The (2,3)-TM is interesting as a simple test case for demonstration simulation of TMs with other computational mechanisms, but I am not going to pay anything but glancing future attention to (2,3)-TM universality questions and the connection with Wolfram's thesis, the principle of computational equivalence. Resolution of these matters seem irrelevant to the particular focus of the Miser Project.

Controversy and Problems of Definition

Ignoring the controversy around the metaphysical aspects of A New Kind of Science, there is now a new controversy around the way this prize was awarded, the definition of the conditions for the prize, and also the conditions to be satisfied under which the (2,3)-TM achieves UTM-hood, demonstration of Universal Turing Machine capability. Considering that the machine has only two states, for a total of six rules with a three-mark alphabet, most of the work is knowing how to prepare the input and then how to recognize "the answer" of any of its computations.

On the Foundations of Mathematics discussion list, FOM, there is ongoing discussion on how universality is to be understood, how one knows that a terminating computation has indeed been completed (there being no halt state), and how one encodes the initial tape and starting condition before setting the machine loose. In particular, it might be unfair to require some other universal machine to create the initial tape for the (2,3)-TM, and it is not clear whether that is the case or not. There is also some concern for the apparent requirement that the tape may require an infinite extent of initial markings and how difficult that may be to accomplish.

This questioning will play itself out. On the FOM list, award-winner Alex Smith, Stephen Wolfram, and others are testing, clarifying, and testing again with a variety of examples and speculations, all designed to narrow down the discussion and conclude how the (2,3)-TM fits into the universality pantheon, and just exactly where.

To help frame the discussion, some useful resources have been pointed out:

Martin Davis points to two historical articles on the definition of Universal Turing Machine.
Damien Woods provides some background and a reference on a variety of small Universal Turing Machines, and the variability in universality conditions.
Vaughn Pratt, in a reply to Alex Smith, provides a nice rundown on the topics of universality and non-halting computations as they have been addressed in what might be termed the mainstream development of computation theory. Pratt, in his cataloguing of contributions, mentions that some have come to advocate more-abstract axioms for recursion theory (the test case for equivalent universality in the Church-Turing convergence). This led some responders to a new topic (below).
On November 17, Vaughn Pratt looks over some earlier discussion and summarizes that "incomparable conventions for small Turing machines yield incomparable results." Pratt goes on to object to certain machines which can be shown as equivalent to being two-counter machines, and hence prospectively universal. [This has me think of this as a rudimentary Turing-Machine counterpart of the von Neumann quality, where two registers (what instruction location, what data location) are prominent, a distant reach from (2,3)-TMs.]
On November 19, discussant Bob Hearn provides some comments from Marvin Minsky and returns to the two-counter machine consideration with "Incidentally, the reduction chain from arbitrary Turing machines to cyclic 2-tag systems to Wolfram's (2,3) machine proceeds via two- counter machines." [This is not, I think, a smoking gun, but something to comprehend more carefully.]
On November 21, Alex Smith proposes a "nested-universality" criterion and I think it is safe to say that what we have here is very much a work in progress and that a sharp universality criterion for the (2,3)-TM is in need of considerable work. This is valuable for those who find the question important, although there is here before us the potent pitfall (and danger to all scientific investigations) of changing the criteria until the predetermined candidate is included.

Although it is not crucial to the analysis of the (2,3)-TM and agreement on the way in which it is to be accepted as universal, the demonstration of such universality is claimed by Wolfram to be evidence for his thesis, the Principle of Computational Equivalence (PCE). Wolfram's argument is that "most systems are computationally equivalent. For example, the workings of the human brain or the evolution of weather systems can, in principle, compute the same things as a computer. Computation is therefore simply a question of translating inputs and outputs from one system to another." This principle should not be confused with the Church-Turing thesis, which does not suggest where computation models are manifest, merely what the limitation of any of their powers seems to be. Wolfram accepts the Church-Turing thesis to the extent that the PCE claims an equivalent ceiling on what these ubiquitous computational mechanisms can achieve and the UTM is its exemplar. [It strikes me as ludicrous to consider it a simple matter to arrange "inputs" --conditions on the planet -- by which the weather system carries out an arbitrary desired computation. It is interesting that there are, at this point, also concerns over how inputs for the (2,3)-TM are to be arranged.]

Kovas Boguta (at Wolfram Research) points out that the PCE is not a mathematical claim but an empirical one so refutation in the mathematical sense is inappropriate. [The same holds for the Church-Turing thesis, but the two theses are not equivalent, as I've already mentioned.]
On November 19, Boguta follows up with a "Principles of Computational Equivalence" that goes into greater detail, contrasting PCE with the Church-Turing Thesis (CTT) and relates PCE (versa CTT) with regard to equivalence between complexity of behavior, equivalence between different computations, and equivalence between computational processes and physical processes. [This for me reveals the degree to which adoption of PCE can slide into metaphysical acceptance of incomparables and incomensurables.]
On November 20, Wolfram Research's Todd Rowland provided additional commentary on PCE and how it is observable: "if you see something in nature that is complicated, then that thing is likely to be universal" and "if you search a space of computational rules starting from the simplest cases, then you will quickly find examples of simple rules that are universal." There are links to further material. It is clear that we will be circling around these informal observations about universality and ponder how one can have a sharp criterion in the face of such widely different empirical settings.

Higher-Level and Practical Computation-Model Formulations

Meanwhile, Richard Heck, noticing Vaughn Pratt's rundown, asks for more about more-abstract axioms for recursion theory. Further contributions accumulate:

Andrej Bauer provides a recap of different ways computability (and computation) theory is being taken more abstract, with a nice collection of thumbnails and references.
Vladik Kreinovich suggests that Abstract State Machines be added to Bauer's survey. Bauer, who is familiar with the model, wonders whether ASMs have been used in computability theory.
Yuri Gurevich responds to Bauer's question via Kreinovich, citing papers in his extensive annotated bibliography that bear on that and related questions.

And we find ourselves at an interesting point of practical application (e.g., algorithms) of abstract (universal?) computation models. I find Gurevich's work to be interesting, especially because of the way it is being applied. (The Miser Project is not anywhere close to being able to address computation at that level.)

Meanwhile,

John McCarthy quietly slipped in a straightforward observation that establishes the simple beauty of practice tied to theory.

[update 2007-11-22: I added some of the latest FOM material and I think that is more than enough. I am satisfied to have shown how the discussion is going (and how this level of scrutiny should have preceded awarding of the prize, just as for the Millenium Challenge prizes. I don't think my interests in universal computation in the limited context of the Miser Project are impacted by this struggle universality and the principle of computational equivalence. The topic of higher-level and practical computation models remains of interest to me for later.]
update 2007-11-20: I added a few more links suggesting how this exploration wanders on and off of unexpected territories.]

2007-11-14: SeaFunc Meet-Up

2007-11-15T22:42:00.001-08:00

Technorati Tags: orcmid, functional programming, SeaFunc, meetup

Four of us turned up for the Wednesday, November 14 meeting of SeaFunc, the Seattle Functional Programming informality. I was here because I said I would. Just in case no one bested the rainy evening, I brought my laptop and started composing some blog posts (including this one).

Daniel, Stephen, and Nico showed up near 8pm and we jawed until 10pm. Nico is the second person at a SeaFunc meeting that owns a Symbolics Lisp machine. He talked a little about his efforts to install a larger hard drive and create a clean base system. Apparently there is a lot of source code available (in Common Lisp) and Nico wants to learn from it. There was some discussion of the power of the IDE and interest in how it might contrast with Smalltalk.

Daniel and Stephen are both with Microsoft. Daniel and I ended up talking about functional-programming a bit and he explained the trampoline idea to me (since I am interested in Henry Baker's Cheney Copy on the run-time stack technique). We managed to include discussion of the One Laptop per Child machine (XO), WHS, Windows Live Services (and the webcasts I've been watching).

I didn't take any photographs. I was having too much fun jabbering. As recompense I offer the following tidbits:

In the November 14 issue of the Dr. Dobb's Report newsletter (don't know that it appears on the web), editor Jonathan Erickson leads off with "Functional Languages: Into the Real World." The signal, to Erickson, is the announcement that Microsoft F# is to "become a first-class citizen of .NET and run on the Common Language Runtime." Jonathan provides a link to the John Hughes paper, "Why Functional Programming Matters."
As evidence of Microsoft's intention, Don Syme of Microsoft Research Cambridge just posted his welcome to Luke Hoban, the Program Manager for F#. Although no promises have been made, it is useful to know that Luke was involved in the creation of C# 2005 Express Edition.
Over on hubFS, the forum for F#, there is regular discussion and advice on running F# on Mono. That means it should not be difficult to have F# (and Moonlight, the Mono counterpart to Silverlight) running on XO. You knew I'd end up here, right?

Miser Project: Repaving Kick-Off

2007-11-10T13:17:00.001-08:00

Technorati Tags: Miser Project, site repaving, web presence, site infrastructure, Progressive Development

I mentioned that there is an important need for housekeeping when I proposed Whither Peano. The site has become seriously out of step with regard to my latest approach to organization and construction. In addition, I need to make a systematic review of the site to correct any broken links that resulted from moving to a site with case-sensitive HTTP resource names. There must be some broken links as well as outdated material.

I will be repaving. It won't have anything to do with the technical content, but it will show up as editorial changes to the content. Also, new technical content will be provided under the new regime.

As repaving progresses, you will see evidence of it in the pages that you visit and especially if you happen to visit any of the construction structure of the site. The construction structure, as for all of my sites, has materials that account for the construction of the site itself. One project in that material will be the repaving project. There will be ways to see what the current stage is and also see where the changes are occurring.

Although it is not critical to the technical content, these changes do support the accountability, dependability, and transparency that I want to provide in every aspect of the Miser Project. There will be occasional progress reports.

This activity will not inhibit the posting of more technical articles on the themes of Numbering Peano. Sometimes I may appear to be A.W.O.L. when I am actually down in the sewers fixing the plumbing.

The model for the updated organization is exhibited in the ODMdev: ActiveODMA Development Framework site's Construction Structure. I shall introduce a corresponding structure and organization for the Miser Project.

What I call Smart Librarianship Rules apply to this change. The idea is that, when changing a system of organization (such as a catalog system), new material (accessions) are brought in under the new scheme, with older materials converted as it becomes important, possibly because of intended new usage or updates. Other changes are left to opportunistic treatment on a whenever-I-get-around-to-it basis.

The first changes provide an on-ramp for the progressive introduction of further changes. For the Miser Project, there is urgent need to revise two major portions of the site on which most other material depends: the front porch consisting of those pages at the root of the http://miser-theory.info/ URL, and the Construction Material section having all of the bits used in carrying out the repaving. The first is the anchor for expanding changes to the construction structure and the content materials requiring the next-immediate attention. The second can be done piece-meal, but it must start out organized well enough that major disruptions in this material will not be coming later. In particular, a document for tracking the repaving itself is maintained in the Construction Material section, which also holds templates used on pages throughout the site.

The first changes only deal with plumbing of the site. Another change, after the plumbing is refitted, is to provide Creative Commons license information on the pages of the site, in the same way as done for ODMdev. I might stick with the Attribution 2.5 for now just to have the changes be accomplished as quickly as possible.

Because the changes are piece-meal, there is a problem about knowing the status of changes in different parts of the site. I also need to tell quickly where the repaving is completed. This is accomplished by leaving placeholders in various construction structure and construction work-item pages on the site. This is for my own sanity in noticing where there are opportunities to do a little repaving while I am working on a portion of the site, and to know that I don't have anything new to do in other places.

My urgency about this is my pent-up urge to create a large number of new items for Miser Project. I need a way to avoid rework for the new material and to accomplish the repaving while there is not so much existing material to repair.

Hark! Is That an Idiom That I See Before Me?

2007-11-07T15:47:00.001-08:00

Technorati Tags: orcmid, Miser Project, programming idioms, idioms, Frugalese, Alan Perlis

"Get into a rut early: Do the same processes the same way.
Accumulate idioms. Standardize. The only difference (!) between
Shakespeare and you was the size of his idiom list
- not the size of his vocabulary."
-- Alan Perlis, Epigrams on Programming (1982)

I mentioned, in Miser Hacks II, that I fancy having abundant idiomatic usage of the Miser Ob structures and the constants that elicit particular operations of the standard Miser machine. This was my justification for having the primitive functions on obs be total in Miser Hacks I.

This is a desirable quality of a machine-language organization and design: having instructions and fundamental data elements that can be used in a variety of ways that allow introduction of unexpected computational flexibility. And, as a practical development, oMiser is a synthetic computing machine. Even though it is constructed atop a conventional digital computer platform, attention to the opportunity for valuable idioms in the "machine language" for Miser matters from the standpoint of economical expression and efficiency of operation.

Alan Perlis woke me up to idioms in programming in a conference-break conversation about work on APL idioms reported with one of his students. In this case, the language was APL, a language whose functional and operator characteristics provided an abundance (including map-reduce and others) of the kind we seek for Miser programming.

A common idiom, one that programmers of the earlier Fortran systems knew by heart is on the lines of the statement

N = I-(I/J)*J

for computing the remainder of the division of I by J. I'm not going to explain this beyond pointing out that it once worked in Python and it still works in C and C++ except now we have the more-explicit

N = I%J;

There are many programming idioms and they are to be treasured. As one can see, idioms can cost something in understandability. Yet when you know the idiom, you can spot it's usage immediately. One which I use regularly is

(I+J-1)/J

for the number of containers of size J that are required to hold I objects.

In a way, programming-language idioms (including machine-language ones) are miniature implementation patterns for common situations. They presage the higher-level understanding of design patterns and they are a significant element in fluency of master programmers.

An example of an idiom in oMiser is a common choice for representing small numbers. In Frugalese, the structure

'(ob-B ob-B ob-B ... ob-B ob-ARG)

where there are exactly n occurrences of individual ob-B (with n = 0 OK), is equivalent to the ob

ob.c(ob-B, ob.c(ob-B, ob.c(ob-B, ... ob.c(ob-B, ob-ARG) ... )))

And that ob (called ob-n for short), taken as a program, has the useful result that

ob.ap(ob-n, x) = ob.b(ob.b(ob.b( ... (ob.b(x) ... )))

where there are exactly n ob.b operations on the right. For those playing along at home, here are related characteristics that are part of why it works that way:

ob.ap(ob-B, x) = ob.b(x)

ob.ap(ob-ARG, x) = x

ob.ap(ob.c(rator, rand), x) = ob.ev[ob.c(rator, rand), x] ob.c(rator, rand)

ob.ev[f, x] ob.c(rator, rand) = ob.ap(ob.ev[f, x] rator, ob.ev[f, x] rand)

-- where no exceptions to the above rule arise in this context

ob.ev[f, x] ob-B = ob-B,

ob.ev[f, x] ob-ARG = x

[Update 2007-11-29: I realized that the original brief sequence was a little too brief. I added some intermediate steps and adjusted the notation a little to at least make this coherent no matter how opaque it remains. I will provide a more-complete definition of ob.ap(f, x) in a forthcoming post, and the "Frugalese" notation being used will also be unfolded in later posts.]

That is, the structure, ob-n is a program (or a sequence) of n ob.c's that has the effect of diminishing any ob (treated as a sequence) by removing the first n beads but no more than there are. Because all singletons have ob.b(x) = x, the technique is assured not to remove a terminal singleton (such as the ob-ARG individual at the end of some ob-m).

That is,

ob.ap(ob-n, ob-m) = ob-(m ∸ n)

where m ∸ n (read: m dot-minus n) is the diminish operation for non-negative natural numbers, resulting in max(0, m-n). [Note: dot-minus is a special Unicode character that may not render in your browser. See if it makes a difference to force the encoding to be understood as Unicode UTF-8.]

This is part of a Miser idiom for small numbers and using them for counting down (and decapitating obs taken as sequences). So we will often see operations of the following sort:

ob.c(ob-B, ob-n) = ob-(n+1)

ob.b(ob-n) = ob-(n ∸ 1)

and constructions that are tantamount to

b-count-of(x)

= ob-ARG, when is-singleton(x)

= ob.c(ob-B, b-count-of(ob.b(x))), otherwise.

When you consider how unnatural Miser is as even a machine language, it is no wonder that without little idioms that are easily reused (and Miser makes reuse amazingly easy) no programs of reasonable complexity could be created "by hand", as when creating the first programs for making writing the programs easier.

Because oMiser is such a rudimentary applicative system, kept that way to see how we can first comprehend and then transcend its puristic sandbox with xMiser extensions, it is appropriate to end with another Perlis epigram (but not the last word on the subject):

"Purely applicative languages are poorly applicable."

[update 2007-11-29: I added some intermediate steps in the ob.ap process so that the evaluation of an ob-n application can be followed, although the unfamiliarity of the Frugalese notation may take some giving up of preconceptions about how to read these formulas. I have to reread them regularly to remind myself that I got it right, and that is what led to this "improvement." I also tweaked one of my sentences about the value of idioms, because I was here and I could.]

Whither Peano?

2007-11-05T16:41:00.001-08:00

Technorati Tags: Miser Project, Numbering Peano, SeaFunc, Chris Diggins, Charles Petzold, Computer Science, Functional Programming, Computation Theory, Henry Baker, Chicken Scheme

I have been very neglectful of Numbering Peano and the other elements of the Miser Project. Although I continue to make notes and follow other activities, I am much too silent and there is no velocity to the work.

I also have other commitments, and those are challenging for me, even though they will train me when it comes time to lay down some code for a reference implementation or three of the Miser engine. I trust that the heavy lifting done there, on practical middleware and its deployment, will provide for a sound implementation of distributed-Miser operation.

A Renaissance

Amid all of that, it is the Miser Project that arouses my passion, even if only for episodes connected to inspiration generated through interactions over the work of others. Charles Petzold's Annotated Turing announcement has been one of those inspiring situations. I intend to take that as a lever for re-launching activity on Miser and visibility on Numbering Peano.

How will I balance that? Every day, the Miser Project will be my after-dinner dessert. It is my reward for completing my other commitments and re-energizing with attention to the Miser Project. I don't know if there will always be visible output, but my intention is that there will be some sort of post almost daily. Sometimes the posts aren't here but on a blog, such as Professor von Clueless, or on a site such as TROST or nfoWare. However it shows up, it will be building toward delivery of working Miser Project software.

I notice that the sand runs out of my hour glass in the evening and I don't have the energy for some of my other commitments. Yet work on the Miser Project is, at this point, energizing. When it ceases to be inspiring and energizing I will re-assess how to maintain the momentum. There are parts that are difficult for me and I will need to be sharp and alert to work on them. That will probably require some fresh-faced morning-hour concentration. I will adapt.

Housekeeping

There are some housekeeping chores that need to be done here. The bits of the site, and the blog too, have some dry rot and repairs are required. Some of this is related to a change of hosting services which, along with big improvements and great economy, introduce case-sensitive folder and resource (web-page) names. See also, 2007-11-10 Miser Project: Repaving Kick-Off.

I need to be diligent about the housekeeping so that the site works with all of the refinements that I have learned and developed on sites that are more current. I want to get that done here while the amount of material is modest. Ultimately, the housekeeping will spill over to my Orcmid's Lair site, since my bibliographic compilations are there. Those have been neglected too, especially regarding materials on mathematical logic and programming systems, especially functional-programming.

Some related housekeeping involves uploading photographs from events that are connected to the work here.

Focus on Content

Speaking of bit rot, the oMiser Sketch is in terrible shape and there are some serious errors in the content, ones that will derail a newcomer to the material. I have also refined my ideas about some parts, and that is being reflected in posts here. But it is time to merge all of my pent-up changes into a refactored, simplified sketch. That will also involve moving material to sections of the site where it can be given focused treatment and expanded supporting development. The idea is that the sketch will shrink to sketch-sized digestible material as an appropriate overview of material that is expanded elsewhere.

I also want to start merging in references to related material, attention to other blogs and work such as that of Christopher Diggins on his cat language and his provision of interesting tools. It will also be interesting to get to the point where I can adapt Henry Baker's Cheney copy technique as it is incorporated in Chicken Scheme, returning me to techniques first explored in the original Heathkit H-8 demonstration of a prototypical Miser implementation almost 25 years ago. These are all pent-up affections that I need to channel along with many of the additional ideas that arise each time I expand on any single one of them. My job jar is already larger just because I finally set down an account of Miser Hacks II.

But What About Numbering Peano?

The last hints about the original inspiration for this blog were published in 2004:

Compiling Num.java 0.1x (2004-10-06) I have much more proficiency with Java now and I need to pick up the demonstration again.
A Fine Kettle of Fish (2004-06-05) I also have a "specification" of Num.java that nails down the contract and also ties it to the Peano axioms for arithmetic.

There is more to be done here, especially without wending such a turtuous route. The idea is to make a very simple demonstration of how we are able to manifest abstractions in the behavior of computer programs, and how interface contracts support that. There is much to be learned from this simple example and its progressive development, and I won't resist the opportunity to resurrect it for very long.

And Focus, How About Focus?

This blog has been a catchall for abstraction- and computation-theoretical material in an odd combination. I don't mind that so much. There is room under the /astraendo marquee for other blogs beside Numbering Peano. I can imagine ones for /Turing, /Church (maybe), and /Gödel, as peer branches for Peano (here, /pn). There could even be a wiki or two. I'm not in a hurry about any of that.

[update 2007-11-10: I added a link to the just-starting repaving activity in the Housekeeping discussion.]

Miser Hacks II: A Hole to Bind Them

2007-11-04T21:28:00.001-08:00

Technorati Tags: orcmid, Andrew Appel, FLoC2006, Miser, Lisp, primitive operations, computational model, abstract data, array theory, axiomatic data structures, Trenchard More

Context

In Miser Hacks I: Floating Along the Ground (2006-08-21), I described how the Miser Ob data type has an equivalent structure to the fundamental data structure of Lisp.

Lisp has a supply of atoms, the cons[x;y] pair operation, and the access functions car[z] and cdr[z].

In Lisp, the structure is completed by equal[x;y] and by atom[x], the "oracle" that informs us when some z is not a cons[x;y] and for which car[z] and cdr[z] are consequently undefined (and, in modern implementations, these are invalid operations). Lisp is defined with another oracular operation, eq[x;y] that for atoms, reports whether or not x and y are determined to be the same atom (or not).

Lisp is computationally universal and profoundly interesting by virtue of a specified universal function by which all computable functions on these structures can be carried out using programs coded in such structures.

Miser uses a similar approach, having a supply of individuals and corresponding functions ob.c(x, y), ob.a(z), and ob.b(z), with "=" for equality among obs. (There is a function ob.d(x, y) that determines equality/inequality within the computational model of the system. That is different than "=" since there are no obs that are recognized as true/false although there are idioms that tend to serve that purpose. That's another hack to be described, and it is an elegant one although I once muddied it by adding more hack than needed.)

The Story So Far

For Miser Ob, the functions ob.a and ob.b are total. Whether or not ob z can be formed by some ob.c(x,y), ob.a(z) and ob.b(z) always determine obs. The Miser individual is the counterpart of the Lisp atom. It is the case that z is an individual if and only if ob.a(z) = z. Technically, the oracular power necessary to distinguish atoms has been moved entirely to "=" in Miser. This works because it is impossible, in Miser, that there be any z such that ob.c(z, x) = z, even when ob.a(z) = z. So we can define our equivalent of atom[z] as simply

is-individual(z) = (z = ob.a(z))

If is-individual(z), it is also the case that z = ob.b(z), but z = ob.a(z) is sufficient to discriminate all individuals.

In Miser Hacks I, I argued that having a structural distinction that discriminates atoms (i.e., individuals) entirely within the system is no more of a problem than having to condition certain statements and derivations depending on the presence of atoms. The guarding of deductions about Miser with is-individual conditions is no different than the need to be guarded with atom conditions in Lisp.

Clinging to Idioms

I claim that having structural features for discrimination is advantageous in the ease with which useful idioms become available, idioms that don't require us to have to be cautious against performance of undefined or invalid operations. So I am clinging to this idiomatic power in Miser. There will be many manifestations of such idioms.

An example of an idiomatic usage of Lisp is in the notion of list. In pure Lisp with no additional types (pretty much ditto for oMiser), there are only atoms and cons pairs. The fundamental data structure is a binary tree. (We ignore the prospect of cycles among cons-pairs, permitted in Lisp but impossible in Miser.) The representation of lists in Lisp is by transiting the cdr-path all the way until an atom (the NUL) is reached. At each point, starting with the whole list, and excluding the terminal atom, the list elements are the car-values at each cons-pair node along that cdr-taken path. That is so useful it is the default sense of the Lisp data structure in common usage. It is the typical representation of Lisp symbolic expressions (SEXPRs).

Singletons and Enclosures

In my preference for idiomatic cases and my yearning for additional representation opportunities, I take advantage of another structural distinction in defining the Miser Ob structure.

Singletons. Any ob, z, for which ob.b(z) = z is a singleton.

Individuals are singletons. In the diagram, the ob identified as x and found at both ob.a(z) and ob.a(ob.b(z)) is an individual and a singleton. This is a structural quality. The names chosen for these qualities has to do with an intended usage, but it is not necessitated by the structure. So we are fashioning idioms and metaphors already.

Enclosures. The other kind of singleton is an enclosure. An enclosure satisfies the condition

is-enclosure(z) = is-singleton(z) and not is-individual(z)

where is-singleton(z) = (ob.b(z) = z)

as expected. In the diagram, ob.b(z) is the enclosure, y, defined as ob.e(x).

Now, just as for individuals, it is impossible for there to be an ob.c(x, y) such that ob.b(ob.c(x, y)) = y ob.c(x, y) = ob.e(x). It can't happen that ob.b(ob.c(x, y)) = ob.c(x, y) ever. That means in theory it is impossible and in practice it is forbidden and prevented: there is no way to achieve such a situation with a Miser computation.

Making enclosures with ob.e. We've managed to finesse where individuals come from (there being guaranteed to be at least one, designated ob-null), and here comes a completely-separate breed of ob. The Miser function ob.e(x) takes any ob and provides an enclosure of it. That is,

ob.b(ob.e(x)) = ob.e(x), ob.a(ob.e(x)) = x

and ob.e(x) = ob.e(y) if and only if x = y

Idiomatically, an ob enclosure can be viewed as a container that wraps up an ob and distinguishes it as a singleton. One might choose to use it, in a Miser application, as a way to separate out enclosures for treatments like individuals rather than a continuation of some represented data structure, such as a tree or a list. Also, notice how one might consider to treat individuals as literals: singletons that enclose themselves (metaphorically of course: no individual is an enclosure).

The use of enclosures for various practical purposes is not determined by the the Ob-structure theory.

Enclosures as Quotations

There is a place where enclosures (and individuals and ob.c constructions) are interpreted in a particular way. That is in the context of the standard oMiser universal function. That function, ob.ap(f, x), and its companion, ob.ev(f, x, p), treat enclosures as quotations. ob.e(x) is Miser's structural counterpart to the Lisp use of cons[QUOTE; x] in the standard Lisp universal function.

To illustrate (but not explain) how this works for Miser,

ob.ap(ob.e(k), x) = k

    but ob.ap(ob.c(m, n), x) = ob.ev(ob.c(m, n), x, ob.c(m, n))

ob.ev(f, x, ob.e(k)) = k

    but ob.ev(f, x, ob.c(s, t)) = ob.ap(ob.ev(f, x, s), ob.ev(f, x, t))

        [ignoring the few cases when s is an individual that marks occurrence of a special operation]

Other features of oMiser and the Frugal processor (which provides a way to input and operate Miser programs) reinforce the enclosure-as-quotation interpretation. It is not the only interpretation, any more than a Lisp occurrence of the QUOTE atom is compelled to have anything to do with quotation in the context of the Lisp universal function.

It is very handy to have a structural form of quotation that is completely separate from the use of atoms or individuals as markers. The use of enclosures in Miser accomplishes that. There will also be use of distinct individuals as special markers in ob.ev operation but those hacks come up later. Other markers are introduced in ad hoc conventions such as ones for ascribing types to enclosures. Those practices are unrelated to the definition of the universal oMiser function and the axiomatic theory of ob structures.

Justifying the Hack

The main purpose of enclosures is to provide a structural mechanism that is easily used to control the depth to which various procedures intrude into the structure of an ob (ob.ap and ob.ev procedures being two examples). Enclosures are a simple, primitive device that supports this need for quotation by structural means along.

There is precedent for a structural quotation feature. It matters in array theory where one wants sub-arrays or sub-something as elements of the arrays that contain them. In this respect, the Miser nomenclature around individuals and singletons honors the array-theory work of Trenchard More.

Another precedent, and the one that I learned first, has to do with stratified strings: strings that can have strings as their individual "beads." This sort of string was defined for ALGOL 60, which introduced a notation for strings as elements (not sub-strings) of containing strings. Although the mechanism did not survive ALGOL 60 where it was seriously under-developed, it offered an existence case for an use of quotation to make string elements out of entire strings.

Christopher Strachey developed a general-purpose macro processor (affectionately known as McG in one implementation) that relied upon stratified strings in a way quite similar to the use of enclosure in the Miser universal function. It was recognition of an omission that prevented universality over such strings that started me on the road that led to the current formulation of oMiser.

Finally, it is useful in theory-focused settings to notice that introduction of enclosures does not in any way extend the cardinality of Ob, the set of all obs. That is, Ob with enclosures is just as denumerable as Ob without them. In this sense, enclosures do not add any inherent power to Miser, yet they simplify the expression of certain kinds of data representations and provide a simpler expression of a universal function. The way that this miniature feature facilitates expressiveness is an useful subject of study and analysis.

Finally, as you can tell, I am infatuated with this feature and will not surrender it lightly. There are enclosures and they harmonize in the play of singletons and individuals and obs composed via ob.c. I like it that way.

Gone Too Far Yet?

What has me look at enclosure as a Miser hack is the degree to which their possibility complicates reasoning about ob structures, the concern that Andrew Appel raised over having ob.a and ob.b be total and defined on individuals. I don't think it complicates reasoning about the universal function much at all. It singles out quotation for the added attention I think it deserves, and however quotation is handled (such as in Lisp) it has to be singled out in the universal function. It is the means by which data is embedded in procedure embedded in data embedded in ... , and it demonstrates that quite simply. Since this is a fundamental matter in computing and its quasi-linguistic character, enclosures seem like an useful idea.

The introduction of enclosures does add cases that must be accounted for in the low-level derivation of proofs about algorithms on obs and the correctness of representations of one kind or another. I haven't found that to be a problem at the low-level so far. I expect that, when one moves up to higher levels of abstraction, any complication due to enclosures becomes irrelevant and disappears. I am open to challenge as I move along. I would be surprised to have to abandon enclosures at some future time.

But Wait, There's More!

There is another hack that qualifies for my ugliest language-theory hack ever. I was proud of that one too until I found the counter-example that defeated the whole enterprise. To show that I am not totally unwilling to reconsider my pet features, I'll show you the hack I'm too embarrassed to keep around even though no substitute is particularly pleasant.

Stay tuned ...

I know it's been a long time since Miser Hack I and the inspiration that I took out of FLoC 2006 and the subsequent ICFP 2006 in Portland, Oregon. I have been working along silently and infrequently. Meanwhile, I on a different commitment unrelated to functional programming and applicative systems such as Miser. Some of that work is useful as preparation for a Miser implementation. I think I have found an appropriate way to break out of my silence and proceed in parallel with that other work.

This break in my prolonged silence was inspired by Charles Petzold's announcement of his forthcoming book, The Annotated Turing. Without knowing anything about how that work will unfold, I feel kinship with the spirit of that effort and this is my offering from the chorus, uh ... peanut gallery, oh uh ... rooting section.

[update 2007-11-12 There is a completely incorrect and misleading statement that I had to repair. I have stricken out the bogus statement and replaced it with an appropriate one around how no ob.c(x, y) can ever be a singleton.]

Petzold Annotates Turing!

2007-11-04T12:12:00.001-08:00

Technorati Tags: Charles Petzold, Alan Turing, Turing Machine, Universal Turing Machine, UTM, Church-Turing Thesis, Computation Theory, On Computable Numbers, orcmid

Charles Petzold, author of highly-regarded books on programming for Microsoft Windows, is turning his attention to a work of love: creating an annotated treatment of Alan Turing's famous paper that set the foundation for a mathematical theory of computation and what it is possible for a computational mechanism to accomplish.

Another highly-respected Petzold book, Code: The Hidden Language of Computer Hardware and Software, illustrates Petzold's devotion to promoting popular understanding of the principles behind computing. Delving into the fundamental theory established by Turing is a welcome sequel. At one point, Petzold lacked a publisher for this project. It is exciting to learn that Wiley is publishing the book, a detour from Petzold's consistent presence as a Microsoft Press author. Wiley Executive Editor Chris Webb has posted more information about the book on his blog. I'm as enthusiastic as Chris is that he is working with Petzold.

The Miser Project comes at the principles of computation from the Church- side (or, more precisely, the John McCarthy side)of the Church-Turing thesis, because I find applicative systems more comprehensible and more easily shown to be applicable to conventional computing problems. But that is not to shun the Turing side of the picture and the contribution it made by giving computation an intense definiteness. In fact, it is always important to understand the mutual equivalence in terms of capabilities and other characteristics, the basis for speaking of a Church-Turing thesis in the first place.

I look forward to Petzold's posting of further details along with tidbits on the by-ways that don't fit entirely into the coverage of the book but do inspire coverage on his blog.

Update 2007-11-22: Petzold has added a section about the book to his web site.

I notice one gift that I've received from Petzold's newest book project. In my excitement over this development, I am committed to tidying up the Miser Project and Numbering Peano for more regular use and usefulness. Consider it my holiday-season present (along with the Windows Home Server that I am gifting myself here in Orcmid's Liar). I have some other commitments around middleware and ODMA. That means I must throw more coal under all of it and get those boilers steaming. I'm overdue.

[update 2007-11-07T11:30 -0800: It appears that Blogger wooziness may have been related to this. It seems to have been fixed.
update 2007-11-05T21:42 -0800: Added the link that I learned from Chris Webb in a comment that has reached me but is not on this page. It seems that Blogger is having some wooziness today. I linked forward to my plan for revamping Numbering Peano too.]

2007-05-02T17:07:00.000-07:00

Hazard Warning: Much Site Breakage

I’ve just noticed how much damage there is on the Miser Project site as a result of my transfer to an Apache server. The previous Windows server did not treat URLs as case-sensitive. On Linux, Apache servers are case-sensitive. This has made problems with cross-references within the site and into the site. I am slowly repairing the cross-references within the site. Bookmarks into the site may end up being broken. The difference is usually around changes in capitalization in URLs.

Fortunately, this is a relatively small site. I will be repair internal links and provide some redirects for frequently-attempted broken external links over time. For now, I just want to offer this warning.

Tags: orcmid, Apache, case+sensitivity, broken+URLs

Tweaking Technorati

2006-11-30T21:33:00.000-08:00

This is a maintenance post. It’s sole purpose is to drive out a fresh main page using an updated Technorati claim. I’ll not explain that further.

I do have a need for the Professor’s assistance though. I need to hack up some tools to change the links that are already embedded in existing archive pages. I make it a policy to never republish the site but to let it reflect the differences that applied at different times in its development. Hence the need to do something more subtle. I have no idea when or how I will get around to that.

Tah

... feeling unlucky ...

2006-10-29T12:56:00.000-08:00

I am having trouble posting to the other three blogs. All four, including this one, posted via Blogger to the new official sites yesterday, but now there is a problem.

I am making an off-topic attempt here to see if the problem is domain-name specific. Let’s see.

[update 2006-10-29-20:51Z Well, this post isn’t going well either. I could try a different ftp account, but I don't think that's it. If this post doesn't make it now, I'll check with Blogger support.]

[update 2006–10–30–01:17Z: Blogger posting via ftp to my official sites is working again, and the appearance of this post should confirm it. There is more of this saga on Professor von Clueless. It is not particularly satisfying to have one more piece of evidence for “It Was the Best of Webs, the Worst of Webs.”]

Hanging by a Branch

2006-10-28T20:27:00.000-07:00

I have changed my Blogger setup. This post will appear at the official Numbering Peano blog location. It will not be reflected at http://nfoCentrale.net/miser/astraendo/pn.

The same is true for the site feeds. Only the official one will carry this post.

There’s a certain amount of frustration with developing on systems that have case-insensitive file systems and moving content to a case-sensitive file system. This leads to odd breakages that are difficult to repair in simple ways. I will persevere, but there will be breakage related to mismatched case-sensitive names that were previously not a problem.