Collaborative articulation of how abstraction and language is employed in the computational manifestation of numbers -- including analysis of the role of syntax, semantics, and meaning in the specification and use of software interfaces.

visitors to popular Miser Project pages

Recent Items
 
Miser: Interpretations of Identity
 
Turing Arrives: Petzold on Arithmetic
 
Reality Is the Model
 
Catching Up with Turing
 
Miser: The Logic of Ot
 
Miser: Frugalese for Applicative Operations
 
SeaFunc: 2008-02-20 Functional Programming Meetup
 
Miser: The Immutability of Obs
 
Computer Science Classics: Your Picks?
 
Cybersmith: How Many Functions Am I Holding Up?

Archives
2004-05-30
2004-06-06
2004-06-13
2004-06-27
2004-08-22
2004-08-29
2004-10-03
2004-12-05
2005-01-23
2005-06-19
2005-08-14
2005-10-09
2006-01-01
2006-02-05
2006-02-12
2006-07-09
2006-07-16
2006-07-30
2006-08-06
2006-08-20
2006-10-08
2006-10-22
2006-10-29
2006-11-26
2007-04-29
2007-11-04
2007-11-11
2007-11-18
2008-01-06
2008-01-13
2008-01-27
2008-02-10
2008-02-24
2008-05-04
2008-05-25
2008-06-08
2008-06-15

Miser: Interpretations of Identity

[update 2008-06-17T16:08Z: I added t0' because I couldn't stand not to show the better refactoring of t0.  I added mention of equivalence classes in an interpretation and also came up with conditions other than invalidity for finding an interpretation unusable.]

In The Logic of Ot, I said that I would use informal expressions of Ot, the logical theory that applies to Miser Obs.   Now that there has been some use of the special characters and notations of First-Order Logic with equality, I want to take advantage of that to talk about interpretations of identity in models of Ot.  The ability to identify and distinguish has great bearing on computational systems, and identity as an interpretation is particularly useful to explore.

= as Equivalence Relation

With FOL=, identity and the relational operator, "=", are taken as given, and the following hold:

=1: ∀x(x = x)

=2: ∀x∀y(x = y → y = x)

=3: ∀x∀y∀z(x = y  ∧ y = z  → x = z)

≠0: ∀x∀y(x ≠ y ↔ ¬(x = y))

The first three are the common properties of equivalence relationships: "=" is reflexive (=1), symmetrical (=2), and transitive (=3).   The final condition is essentially the definition of "≠" in terms of "=".

What Are We Talking About?

In a first-order logical theory, the variables (x, y, and z as seen in ∀x∀y∀z) are understood to refer to objects in the domain of discourse.  We only know what there is to know about that domain from the introduction of constants and expression of conditions that are theoretically required to be satisfied over that domain.  For FOL=, we are given an equivalence relation (expressed with the symbol "="), which tells us very few things about conditions under which variables can be taken as referring to the same object of the domain of discourse.

It should be apparent that having "=" doesn't tell us much about the theoretical objects, although it is more than not having "=" (and its partner, "≠").

An intended interpretation could well be that objects in the domain of discourse be identifiable (we can tell when we are referring to the same one) and discernable (we can tell when we aren't).   Let's see how that might work.

A Tiny Domain

To provide some practice with ideas of practical interpretation, consider the logical theory obtained by adding the following conditions:

t0: ∃x∃y∃z∀u( x ≠ y ∧  y ≠ z ∧ x ≠ z ∧ (u = x ∨ u = y ∨ u = z) )

or the equivalent,

t0': ∃x∃y∃z( x ≠ y ∧  y ≠ z ∧ x ≠ z ∧ ∀u(u = x ∨ u = y ∨ u = z) )

The intended reading of this is as

1. There are at least three different objects in the domain of discourse, and
2. Any object in the domain of discourse is one of those

which is to say, there are exactly three objects in the domain of discourse.

If we only have that one additional condition (t0) in our logical theory, we know nothing beyond that. 

Notice that we have not labeled the three theoretical objects in any way.   All we have provided for is that there be exactly three.

It will be useful to appear to be more specific by naming them:

t1: A ≠ B ∧  B ≠ C ∧ A ≠ C

t2: ∀u(u = A ∨ u = B ∨ u = C)

Here, A, B, and C are constants for objects in the domain of discourse.   We haven't provided much more than what (t0) assures us of, although if there is more to say about the ways that the three objects differ, having the constants to refer to may be useful.

Illustrative Interpretations

The first interpretation will be in terms of numbers.  Assume the system of numbers and arithmetic.  Take that system as separate from the logical theory consisting of FOL= plus t0 and t1.  (t2 is actually a consequence of that much.)

One interpretation of our tiny theory in number theory would be by saying that the interpretation of A is any n < 0, the interpretation of B is 0, and the interpretation of C is any n > 0.   We could be specific, say, with interpretation of A as -1, B as 0, and C as +1.   We could also say that A is all n < 0, B is 0 only, and C is all n > 0.  That is, A, B, and C correspond to distinct classes.  Since they have no members in common, these are known as equivalence classes.  We'll explore that further when we return to exploration of Ot.

It doesn't matter, here, how the interpretation is chosen, so long as, having made it, we stick to it.  The system in which we make the interpretation is a model (in the loose sense of Reality is the Model) provided that all deductions in our logical theory hold in the interpretation.

Because the logical theory says nothing about aspects of the model that are not accounted for in the logical theory, those matters are irrelevant to the conditions of the logical theory.  It does not matter how many different ways the interpretation could be made in the model, so long as when one is made, the logical theory is seen to hold for the interpretation.  There is a common fallacy involving reasoning about extra-theoretical characteristics of the model to argue that the theory is incorrect or inapplicable, when the disagreement is more-appropriately viewed as one over choice of interpretation.   It helps to carefully separate the theory from its interpretations and models to avoid that pitfall.  The abstract theory and the logical formalism is helpful in that regard, even if it feels quite unnatural.

"="/"≠" Are Interpreted Too

It is easy to overlook one important feature of an interpretation of our tiny theory: There must be an interpretation for "="/"≠" in the model as well.  That comes along too easily in our choice of interpretations in the system of numbers and arithmetic and it is easy to overlook.  When we dig into computational systems and the details of Miser, the ability to discriminate "="/"≠" in particular interpretations becomes very important.

Finally, as an interpretation in reality: let A be earth, B be wind, and C be fire.  This is a questionable interpretation quite apart from the omission of water.  The difficulty is assuring that these are cleanly distinguishable concepts.  What do we do with flaming molten lava and the sucking wind of a forest fire?.   We will stumble here at least in an effort to have well-determined "="/"≠" and understandable communication of the conditions that others can accept and apply.  The simple, practical conclusion may be that the interpretation is invalid (or simply meaningless/useless) and the theory is inapplicable in that case. 

In other cases, a certain conceptual sloppiness, if carefully circumscribed, may be tolerable in having useful interpretations in reality.  It remains to be seen whether that is ever very workable.

posted by orcmid at 6/16/2008 11:54:00 AM

Turing Arrives: Petzold on Arithmetic

previously:

2008-05-26: Catching Up with Turing
2007-11-22: More Annotated Turing
2007-11-04: Petzold Annotates Turing!
   

related:

2007-11-17: Different (Universal) Computation Models
2008-02-29: Miser: Frugalese for Applicative Operations
2008-05-06: Miser: The Logic of Ot
2008-05-29: Reality Is the Model

I pre-ordered my copy of Charles Petzold's The Annotated Turing on November 22, 2007.  On May 24, I had to authorize a delay in the estimated ship date.  I waited patiently.  When I saw that Petzold had his copies, I wondered if my order would fall through the cracks.  The amazon.com site listed the book as in stock, but I had no word.  That was resolved this Monday, June 9, when I received notice of shipment.  The book arrived two days later by postal mail.

The problem with actually having The Annotated Turing in my possession is deciding when to start and clearing the time to do it.  I did start reading at the end of the book, and I have nosed into a few other sections.  Naturally, the book arrived at a moment when all of my projects are behind and I am already starting an important new one.  A systematic reading is yet to come.  I know I will love it if only for the historical threads and connections that Petzold traces in the book. 

As part of the tracing of connections, Petzold has been reading One to Nine by Turing's biographer, Andrew Hodges.  There are numerous connections traced there, and I like it that Petzold finds himself arguing with Hodges as he works through the book.

Yesterday, Petzold comments on Hodges' objection to memorization of arithmetic with recognition of his own experience in learning the multiplication tables.  The interesting idiosyncrasy is how Petzold failed to have automatic memory of certain multiplication combinations and he would solve those cases by algebraic deduction when needed.  That resonated for me.  There are many cases where I did not remember a rule, but I could and did recreate it on demand.  I also share Petzold's having done that long after simply memorizing the result would have been more productive.  (This shows up in other activities of mine too, including re-inspection of already-written code to remind myself that it is sound and what the context is before adding more to it.) 

I wonder how much this ability to have abstracted an applicable principle (in my case, remembering the times-11 and times-12 cases in terms of times-10 plus times-1 or times-2) leads to algebraic facility and the handy use of identities and mathematical induction well before I developed anything like a fundamental understanding of number theory over the course of my adult years.  I recall re-derivation as being valuable in test-taking and yet it is not as direct as having embodied the result for immediate availability. 

I can't tell you how many times I have verified for myself what the correct formula for the sum of the first n integers is by redoing the constructive derivation.  My doubt is always between n(n+1)/2 and n(n-1)/2 and it is, of course [easy for me to say], the former.  I say that not because I have memorized it but because I know how to tell quickly another way, the first way I ever saw it "proved."  I suspect that I have just sped that up for myself by looking at it anew this time.  Then there's the one about the sum of the first n powers of 2 and what it looks like in binary, etc.  I suspect that our diminished respect for the teaching of arithmetic and how to verify arithmetic results is causing trouble for students and their teachers when it is time to approach algebra where one can't avoid dealing with ratios and fractions by using a calculator.

Seeing this latest post from Petzold has me thinking of the connection with a recent paper by Keith Devlin (via Richard Zack), "The Useful and Reliable Illusion of Reality in Mathematics."  Two connections that come to mind: how we might come to exercise our capacity for abstract, conceptual thinking as we develop our facility with language, and the tendency to see mathematical conceptions as real.  In the second case, Petzold has observed that Turing's machine was his idea for a "real" computer, and I am surprised by that.  There are deeper connections in the Devlin paper with how we end up regarding mathematical objects, and that is worthy of separate discussion with regard to what makes computers so successful and so devilishly difficult to deal with.

posted by orcmid at 6/14/2008 11:00:00 AM

Reality Is the Model

[cross-posted to Orcmid's Lair.  This is at a level of abstract speculation that is more appropriate here than there.  However, I would like a broader audience, and reactions, to what strikes me as having practical importance in how we develop successful computer-based systems.]

During my regular Tuesday buddy call with colleague Bill Anderson, it suddenly occurred to me that I could account for reductionism, an error that scientists and others (software technologists and their masters, for example) make.  It is all captured in the following statement:

 Theories don't model reality; reality is the model.

I don't recall what prompted my exclamation on the subject during the call, unless it was something about how objectionable "code is the model," "code rules," and other developer slogans are, where the implementation of something becomes the specification, denying us access to any useful answer to the question "implementation of what?"

Now, it is not a new thing for Bill and I to be discussing these issues, including this view of the role of theory.   What hadn't landed so sharply was how viewing theories as models for reality is the very pitfall that engenders reductionism.

There's much more careful development required as part of arguing for the usefulness of "reality is the model."   I have been looking at that in a setting where I am conducting a theory-driven implementation of some software in the Miser Project.  Bill and I discuss this even more where it matters a lot to information technology, in contrasting What Computers Know with what Programmers Know to Do and how there is an essential gap between how computer systems are built, the requirements that those systems are meant to satisfy, and the world of opportunity in which those systems are instruments of human purpose.

For now, I want to look at the statement in the context of how we appear to arrive at theories and then apply them as given.

A word of warning: the value of "reality is the model" is not that it is "true."  The value to be found is in having a more useful and powerful way of looking at what we do with theories in contrast with the limitations of imagining our theories to be modeling reality.

Where Theories Come From

There seems little doubt that theories started out as explanations of the regularity in our experience of reality, of the world.  Some of these theories were, and still are, very pre-scientific (as theories about theorizing might be as well, and that won't stop me).

At some point in the course of the scientific revolution, say around 1600, typified by the work of Frances Bacon, there was an important move to development of scientific theories via inductive generalization from observations of nature, not deduction from some principles of cause.  The reliance on experimental confirmation and empirical observation became important.   A consistent case of contradictory results could show where the theory is inapplicable or even completely incorrect.  One risk is that expression of a generalized (abstracted) theory might be taken as an explanation of the nature of nature as in "objects at rest tend to remain at rest." 

Emergence of mathematical sciences, illustrated in the achievements of Isaac Newton, had a profound impact.  It permitted the deduction of consequences by calculation or proof, and it permitted the experimental confirmation of those deductions by natural experiments.  Notice, however, that the deduction occurs inside the theory, as it were, and the correspondence of the conclusion with reality is an empirical matter.

Theories on Their Own

The mathematical formulation of important theories, and the computational applications of those theories, are removed from reality.  Once we are operating in the formal, mathematical theory of a science, there is no reality there.  The connection to the reality is accomplished by our interpretation of the mathematical theory as being about reality.  Being about something, especially reality, is not a feature of mathematical systems.  Being about something is how we interpret results in the mathematical formulation as applying to the world in accord with a scientific thesis.  In other words, the scientific thesis part is not expressed in the mathematical formulation.  That is what we add ourselves (even though it is what led us to the formulation and why it might be of any value to us).   Some of these interpretations have been so reliable and so useful, we tend to speak of the expressions of the theories as laws (E = mc² being a popular one, force being proportional to rate of change of momentum being less familiar, although we experience its confirmation every day). 

There is a combination of deductive process (predicting via calculation, say) and inductive formulation in this approach.  We might say that (empirical) experience has shown that the interpreted deductions are reliable and that the theory is a good one in that sense.

The pitfall is to think of the theory as the truth, as somehow explaining how it is that the interpretation of findings in the theory align.  Perhaps the most current conceit of this nature has to do with confusion of what nature does as computation because computational processes have some similar characteristics.

Interpreting Theories and the Reductionism Pitfall

There is an area of mathematics called "model theory" or what, here, we might call the model-theoretic view of mathematically-expressed theories. 

In model theory, the idea is that a mathematical theory, expressed in a formal, logical way, is given an interpretation by identifying its mathematical elements with those in some other system (usually some other kind of mathematical one).  If the interpretation is such that deductions in the first theory have results that are true in the interpretation, we say that the interpretation is valid, and that the interpretation is a model of the theory.  The model satisfies the theory.  I am omitting many technicalities (and probably abusing model theory) in order to appropriate the basic idea for application to interpretations in the world in mathematical sciences.

An important feature of this view is that the theory need not account for everything in the model.  For the model to be a model everything that is true of the theory is true in the interpretation, but the model is not otherwise constrained.  The interpretation is only for that aspect of the model that corresponds to the theory.  There may many other features and aspects to the model that are simply not captured by the interpretation (and hence the theory).   Under the particular interpretation, at least, however valid (in either a mathematical or an empirical sense, as the case might be), the theory has nothing to offer about those other matters.  In particular, we are free from concluding that the theory explains the model or dictates its "working."  We are also free, in the case of the world and many mathematical and logical theories, of having quite different interpretations in reality for models of the same theory.  (Interpreting objects and phenomena as numbers is something we are able to do in innumerable ways.  I had to say that.)

The reductionist pitfall is treating the theory as the model (and therefore comprehensive), and claiming the theory to be "about" the world.  In that case, there is no way to countenance there being anything else about the world and even the obvious becomes inaccessible.  There's some other kind of pitfall in faulting a theory for not embracing all of reality in its interpretations, but I am less concerned about that, although "reality is the model" avoids that too.

Computational Manifestations of Theories

This apparently-backward way of looking at theories is about the application of theories in ways that are useful in approaching reality.  The perspective is also contrary to using computation as embodiments of theories and seeing them as somehow modeling the world.  Theories may have computational models (as interpretations).  This doesn't make the computation model a model of the world any more than the theory is, in this view.  I say that there is a computation model of the theory, and there may be an intended interpretation in the world that is a model, but that does not make the computational model a model of the world any more than the theory is.

I find this a very fruitful way to look at a variety of aspects of information technology as it is developed and used.  My continuing duty is to articulate this value in less-abstract and directly valuable terms.  One curiosity is how this view can still allow for the notion that the act of programming a computer is a case of theory building.

posted by orcmid at 5/29/2008 02:46:00 PM

Catching Up with Turing

[update 2008-05-29T16:15Z: Added Petzold's 2008-05-28 post with its nice tie-in of logic and Turing.]

 update 2008-05-27T16:03Z: A little tweaking in my comment on Petzold's 2008-05-25 essay.]

I received an amazon.com e-mail warning announcing that Charles Petzold's The Annotated Turing is delayed and I needed to approve the shipping delay from May 23 to June 23.  A number of times, I will receive one of these announcements only to have it followed by my order shipping immediately.  Since Petzold's site reports the book is scheduled to launch on June 16, I will be content to wait. 

Meanwhile, Petzold has been posting interesting tid-bits that attracted his attention while working on the book.  In many cases, there will be deeper coverage when the text is available.  Here are some that I am particularly keen to dig further into:

2008-05-28: Babies Are Illogical: The "Lost" "Chapter" of "Code"

Starting out with a look at logical puzzles, Petzold looks at questions about truth, how connected to logic, and related challenges that did not make it into his book, Code: The Hidden Language of Computer Hardware and Software (recommended).  In this essay, Petzold illustrates Charles Dodgson's 1896 Symbolic Logic approach using George Boole's original 1854 notation and will tie it back to computability in Chapter 12 of The Annotated Turing.
  

2008-05-25: Reading Brian Rotman's "Ad Infinitum..."

There is an interesting tension between computations that do not stop but in the limit ( as we were inclined to say when I was a school student) the output approaches converges to some computable real number.  Turing allowed such cases, and then demonstrated that there are still far more real numbers that cannot be computed than those that can.   The way this is arrived at leaves some questions about how many real numbers we think there should be and whether we are mistaken in where conventional set theory takes us in that regard.
   Although I am not ready to concede to Petzold that any kind of Platonic commitment is necessary in conception of the transfinite, I definitely find this accompanying statement worthy of careful appraisal:
   "We like to pretend that mathematics is the most 'objective' and least human-bound intellectual endeavor, but our view of the natural numbers reveals mathematics to be founded on a very human metaphysical conceit. The natural numbers are not, in fact, "natural" — that is, intrinsically part of nature — but arise out of human discourse."
   In this essay, Petzold explores Brian Rotman's effort to avoid conception of the completed natural numbers (and certainly, in that case, the completed reals).  The focus is on tying Turing's work to philosophical issues regarding the foundation of mathematics, an unexpected connection and one that Turing did not explore over-much.
   

2008-05-22: Turing and Brouwer: The Unexplored Connection

Not expecting to find a connection between the work of Turing and L.E.J Brouwer, Petzold was startled to find one in the short Correction that Turing made to his original paper.  Accounting for this leads to a chapter on "Conceiving the Continuum" in The Annotated Turing and this delightful essay on how that all unfolded.  This is more on the issue of infinitudes.
  

2008-05-18: Turing Machines that Run Forever

From the beginning, Petzold has intimated that he wants to illuminate the Turing Machine as Alan Turing conceived of it, with it being acceptable for the machine to run forever.  This is not the ordinary formulation that has survived into contemporary computation theory.  Turing wants to bring the full Turing characterization to our attention.  This essay motivates the difference and the historical context at the time of Turing's formulation and the subsequent revisions.
   

2008-05-10: The 300 Page Ideal

Considering some of the later blog posts, it is amusing to see this use of "ideal."  This is a wonderful essay on the quirky fascination with attempting to produce a 300-page book, since Petzold has never managed to keep one that short. 
  

2008-05-09: Letting Go of the Book

This is a touching essay on The Annotated Turing now being on its way to print, with no more opportunities for changes or misgivings.  We are reminded that Petzold had been working on the book since 1999.
  

2008-03-27: "The Annotated Turing" Typographical Triumph

Petzold provides an account of what he went through to reset the Turing paper so that he could match Turing's symbols and typeface when elements of the Turing paper are discussed in the "annotations."  This is something that Donald Knuth will love, since the desired result was obtained (as the editor confides in a comment) by using LaTeX.  It is delightful how the reset Turing paper was proofed against the original printing.
  

2008-03-01: Of a Book Entitled "Mr. Turing's Computing Machine"

Petzold provide an account of his 1999 inspiration to produce The Annotated Turing, and the project's early life.

posted by orcmid at 5/26/2008 08:17:00 PM

Miser: The Logic of Ot

In the narrow sense, logic is the theory of valid arguments
or the theory of deductive inference.
  A slightly broader sense includes the theory of definition. ...
-- Patrick Suppes, preface to Introduction to Logic [5:p.xiv]

Miser demonstrates a model of computation.  The specification of Miser establishes the functional requirements for a computing machine (the mechanism).  The machines are typically implemented by software programs operated on conventional digital computers.   That this can be done at all is a demonstration of how well digital computers are useful for manifestation of abstractions.

While it is appropriate to think of a Miser as a machine with a variety of physical realizations, there is a mathematical theory that dictates the essential characteristics that each realization is expected to manifest.  This is by design.  We want Miser to be amenable to mathematical reasoning and analysis. The mathematical theory determines the correct behavior of Miser implementations.  Even so, no result in the mathematical theory can ever be a proof about a Miser implementation.   We want to develop an appreciation for how this is so and how there remains an useful connection between theories and concrete realizations even though the connection is a bridge that the theory can never cross.

The mathematical theory is essentially pure logic applied to a simple subject matter: the Miser Obs and functions over them.  It is also the case that Miser procedures perform in logic-resembling ways and can achieve what is known as a computational logic.  Both of these conditions require that we be careful about our demands on logic and how they will be expressed.  We need to look at how the theory for Miser is expressed as a logical theory as well as how logic is expressed in Miser procedures (that is, by computational means).

This reliance on a logical theory will assist our differentiation among computation, logic, and mathematics.  This will, in turn, help us clarify the relationship between procedures and functions and algorithms for functions as we look more closely at the Miser computational model.

The logical theory for oMiser, the foundation system, is symbolized Ot, short for Ob theory.  It is an application of First-Order Logic.

1. First-Order Logic

The logical theory for Miser is mostly expressed in an informal style.   Most of the assertions under the theory are in the form of equations or identities: equalities and inequalities. 

Although there is not much need for a full-up expression in a formal logic, there is always an equivalent formal expression using First-Order Logic with equality (FOL=).  The FOL= logical symbolism employs the following forms:

x = y equality: read as "x equals y"

x ≠ y inequality: read as "x not-equal-to y"

∀u(p) universal quantification, read as "for all u, p"

∃u(p) existential quantification, read as "there exists u such that p"

¬p negation: read as "not p", also seen as "~p" and "-p"

p ∧ q conjunction: read as "p and q", also seen as "p&q"

p ∨q disjunction: read as "p or q"

p → q conditional: read as "if p then q" or "p implies q", also seen as "p⇒q" or "p⊃q"

p ↔ q biconditional: read as "p if and only if q" or "p equivalent q", also seen as "x⇔y" or "x≡y"

A common order of precedence of the operations is indicated in the progression.  The components of the equality and inequalities can be constants, variables, and expressions involving functions of operands that are the same kind of components.  These are not applicative expressions and should not be read as Frugalese applicative operations (just yet, if ever).

A basic introduction to FOL notation is available on-line in the MIT Open Courseware [2].    There are other on-line descriptions [4, 6].  These should provide enough background to be able to read the formalisms of Ot.

The specific form used here is that described in the Handbook of Mathematical Logic [1].  Leisurely expositions with many examples and exercises are available in the books by Lemmon [3] and Suppes [5].

2. Informal Expression

The informal expression allows us to avoid heavy use of special symbols.  For example, an axiom of equality,

∀x∀y∀z (x = y ∧ y = z → x = z)

could be expressed informally for Ot as

"For Obs x, y, z, if x = y and y = z, then x = z."

There are further examples to be found among the posts on Numbering Peano and in other Miser Project materials.

3. Ot as a Logical Theory

For Ot, there are additional constants to be introduced (e.g., ob_A, ob_B).  We also presume a variety of functions (e.g., ob.a, ob.b, ob.ap).  These and other additions to FOL=  constitute Ot. 

In particular, we assume that the variables of quantifiers (x and y in ∀x, ∃y) refer to Obs.

Because the domain of discourse, as it is known, is the Obs and only the Obs, how functions are represented in Ot becomes an interesting topic.  The same is true for additional predicates (i.e., beyond the predicate implicitly associated with "=").  We will explore these matters in further posts and articles of the Miser Project. 

For now it is useful to point out that there is no direct way to express something about all/any functions or all/any predicates using FOL= (a consequence of being first-order).  When we speak in such a way, it will have to be informally and outside of Ot.  That situation is also to be explored further.

---

[update 2008-05-07T17:29Z: I couldn't resist changing the title from "The Logic of It" to "The Logic of Ot," a bad-pun opportunity which I hadn't noticed until making a diary entry about having completed the post.  It is an apt phrasing, besides.  As long as I am here, I also append a little more content to section 3.]

    

[1] Jon Barwise: An Introduction to First-Order Logic.  Chapter A.1 (pp. 5-46) in Handbook of Mathematical Logic, Jon Barwise (ed). Studies in Logic and the Foundations of Mathematics, volume 90, Elsevier (Amsterdam: 1977).  ISBN 0-444-86388-5 pbk.

The concerns I have expressed over Handbook do not apply to this chapter (although I just found a typo in the first example in section 3), which is tidy and makes useful connection to model-theoretic considerations.  Our notational symbols are those used here.
  

[2] Leslie Kaelbling: First-Order Logic (lecture notes), Lecture 5,  MIT Course 6.825 Techniques in Artificial Intelligence, Fall 2002.  Open Courseware PDF file.  accessed 2008-04-17.

These notes provide a leisurely and elementary introduction to FOL= notation with understandable applications to every-day logical situations.
  

[3] E. J. Lemmon: Beginning Logic.  Van Nostrand Reinhold (Great Britain: 1965), modest revision edited by George W. D. Berry, Hackett Publishing (Indianapolis: 1978).  ISBN 0-915144-50-6 pbk.

One advantage of this text, considered by its author to be more elementary than Suppes[5], with which it matches its notation, is that it is still used as a textbook and the author's insider remarks are entertaining.  The Bibliography notes are recommended.
  

[4] PlanetMath: First-order logic.  (article) accessed 2008-04-17.

This is a good place to find a summary of the notation without all of the subtleties and accompanying discourse.  For a deeper look into how formal logic and logical theories work, Lemmon and Suppes will be rewarding [3, 5].
  

[5] Patrick Suppes: Introduction to Logic.  Van Nostrand Reinhold (New York: 1957), Dover unabridged republication (New York: 1999).  ISBN 0-486-40687-3 pbk.

Recommended for its gradual progression and build-up of first-order logic with useful examples and exercises.  Chapter 11, on Functions, is a valuable bonus (as well as slightly contrary to what we have to say about functions in the context of the Miser Project).
  

[6] Wikipedia: First-order logic.  (article)  accessed 2008-04-17.

This article appears designed to frighten the uninitiated.  It can be useful for its sources.

posted by orcmid at 5/06/2008 07:58:00 PM

Miser: Frugalese for Applicative Operations

[update 2008-05-06T01:23Z: I corrected a small typo before linking to this post from another one in this series on Miser notions.  I also included a reference to the work of Church where he introduces f(x, y) as equivalent to (f x) y.]

 update 2008-04-12T17:49Z: I am still niggling with this post.  There will be a separate post on some of these ideas, but I wanted to add something about being committed to treating functions as definite mathematical objects and viewing procedures as algorithms for (some of) them, even though we don't have a way to uniquely identify and distinguish functions (or procedures, for that matter).  I think I'm content with this for now.

 update 2008-04-12T02:45Z: After completing this post, I realized that all expressions that stand for functions are ones that characterize functions, but none of them "are" the function.  I have revamped the post, today, to emphasize the introduction of functions by characterizing them.  This makes some of the prose very awkward, especially if you just wanted to learn Frugal and use a Miser engine to produce something useful.  It seems like a necessary foundational exercise for me, to ensure that I have successfully wrestled the relevant concepts to the ground.  I took this occasion to do some tweaking to make some other points more clear, although adding the fuss about functions only being known by their characterization is probably a net loss. -- dh.]

On occasion, I mention that Miser is an applicative-programming system.  There is a connection with the notion of functional programming.  I want to clarify that terminology and, especially, illustrate the impact it has on notations for programming in applicative style.

1. The Basic Idea: Application as Fundamental
   1.1 The logico-mathematical view of functions
   1.2 The λ-calculus view of applicative functions
2. Be Ye Operand or Operator?
3. And in Frugalese?
4. Streamlining Applicative Notation
5. What About Multiple Parameters?
6. Left-Right, Left-Right, (Cha-cha-cha) Right-Left
7. Ah, Sugar, Sugar

1. The Basic Idea: Application as Fundamental

The key idea of applicative systems is that the fundamental operation is one of application of functions and their arguments or operands.  This also involves the notion that functions are definite (mathematical) objects.

Consider, for example, the familiar notation f(x) from mathematics.  In mathematics, this notation tends to be used

1. when the function is being used (that is, mentioned) in some larger expression and
     
2. when it is being described or characterized in terms of some essential relationships:

(1)    ( f(x₂) - f(x₁) ) / (x₂ - x₁)

(2)    f(i) = f(i-1) + f(i-2); i > 1

        f(1) = 1; f(0) = 0  

illustrating the respective cases.  In (2), f(n) is known as the n-th Fibonacci Number; we will allow ourselves to say that function f determines the Fibonacci Numbers.

1.1 The Logico-Mathematical View of Functions

In logic and mathematics, the mixture of usages to define/represent/characterize a function and to appeal to a function is commonplace.  It is generally rare to consider f as identifying the function itself, as if functions have an independent existence, with f(x) as separately identifying an application of it. 

In modern mathematics (i.e., that of the last 100 years or so), it has become useful to consider functions more directly.  It happens

1. when defining the type of the function and
     
2. when characterizing the function in the form of an independent entity:

(1) f: N → N

(2) f = λi.(if i > 1 then f(i-1) + f(i-2) else 1)
    
(3) g = Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1)

The form (1) asserts that the function f has its domain of arguments in the natural numbers, N (according to the usual convention),  and its range of results (→) is also in the natural numbers, N.

It is very easy to think of functions as depicting operations that produce something, thanks to the prevalence of computation in modern life, in teaching, and in our learning arithmetic (computation) before algebra and higher mathematics.  Programming languages use "function" in their codes and their descriptions in a way that adds to this collapsing-together of functions, algorithms, procedures, and program code.

A more abstract way to think of individual functions is as specific relationships or correspondences of each distinct member of the domain with at most one member of the range.  An element of the range may stand in the functional relationship to multiple members of the domain, but never vice versa.  It is also possible that a function might not be defined at all, have no corresponding member of the range, for a given member of the domain.

Although this sense of functions as correspondences is perhaps more contemporary, our ordinary way of speaking is in terms of actions that determine results from given arguments.  It is easy to add presume causality, speaking as if functions produce members of the range from members of the domain.   The notations used in computer programming encourage this confusion of relationship and action or procedure.  I notice that I tend to speak of the member of the range being determined by the function together with the selection of a member of the domain. 

Computer folk can also think of this correspondence/relation characterization of functions as being realized by a database table with unique keys, each row having a field for the range value that is determined for the domain value represented by each key.  This works in the case of finite domains and ranges, but we need something else when there is no such limitation (as is often the mathematical case).

In all of the forms that we've shown, the function is not being exhibited, it is being characterized.  There are many characterizations of the same function and many different ways to characterize functions (as we are seeing here).  There is no way to see or get your hands on "the function" because the concept is, in effect, abstracted from the relationship that it embodies (abstractly speaking).  Nevertheless, we seem to be able to grasp what many functional relationships are from succinct characterizations of them.  We rely on functional relationships in day-to-day use of mathematical entities (such as numbers) without having to think about all of this very much.

I'm bringing up this cloudy subject now because digging into the foundations of computing require that we have a sharper notion about functions and what it takes to trace the relationships from domain to range by computation.

1.2 The λ-calculus View of Applicative Functions

The form 1.1(2) involves a λ-expression (pronounced lambda-expression).   This particular one, interpreted as the characterization of a function,  happens to refer to the function that the expression is intended to characterize (singled out on the left) in the characterizing expression itself (on the right).  The given characterization of f is recursive.

The form 1.1(3) specifies the same function in a way that has the characterization be completely accomplished in the right-hand side expression, with no appeal to the name of the function on the right-hand side of the equation.  The special Y-operator is a function which transforms other functions into ones that operate recursively.  In this case,

g(n) = (Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1)) n

        = if n > 1 then     (Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1))(n-1)
                                 + (Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1))(n-2)
                       else 1

illustrating that applications of functions having functions as operands can transform functions to other functions (as characterized).

The λ-calculus is important in Frugalese and for Miser.  It is used here to emphasize that there are ways to characterize functions via self-contained λ-expressions.  These λ-expressions can appear anywhere in the left or right of application operations, whether as functions, as in

(Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1)) n

or as operands, as

λfλi.(if i > 1 then f(i-1) + f(i-2) else 1)

is in

Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1).

The use of "=" with regard to formulae that characterize functions is a bit different than the "=" where we tend to think that a unique determination is happening.  To remove any confusion between uniquelly-expressible mathematical entities (such as natural numbers) and the more-ellusive functions, we could have used "≡" or "∽" or some other indicator of equivalence. We won't be so fussy. So how do we determine that two characterizations that are not identical are still for the same function?

The usual approach is some version of the following procedure.

If it can be shown that f(x) = g(y) whenever x = y, then f = g, and vice versa.  (We are begging some questions here, but that is the gist of it.)

2. Be Ye Operand or Operator?

What you shouldn't miss in the above, is this:

In applicative systems functions are treated as definite mathematical entities that are equally usable as operands (something a function can be applied to) and as functions (something that can be applied to an operand).

That is,

λfλi.(if i > 1 then f(i-1) + f(i-2) else 1) g

for some g: N → N

characterizes the same function as whatever

λi.(if i > 1 then g(i-1) + g(i-2) else 1)

does.  And

λgλj.( g(g(j)) ) (Y λfλi.(if i > 1 then f(i-1) + f(i-2) else 1))

characterizes a function that is very nasty to compute (using the formula as a rule of computation) for increasing values of its natural-number operand.

By having functions be first-class candidates for use as operands and as functions, we have great freedom in characterizing functions, including in terms of  application of functions in the establishment of other functions.

Although this seems pretty esoteric, it is useful to keep in mind that programming languages and compilers are literally embodiments of functions that derive (procedure codes for algorithms for) functions from their data (the source-program texts).  There is a deep connection with some of the fundamental power of digital computers, and Miser is one way to explore that.

3. And in Frugalese ... ?

Frugalese is a language for expressing computations carried out by applicative systems such as Miser.  Miser is a computational system, not a mathematical one.  There is a mathematical connection, and that motivates our giving so much attention to the business of characterizing functions and what that does and does not accomplish.

For Miser and Frugalese, the fundamental operation is called "apply."  Apply has two operands.  Both operands are Obs.  The first Ob is interpreted as a coding for the procedure to be followed.  The second Ob is the data that the procedure will manipulate in some way. 

We need to be clear, here, about what we mean by computation of a function.  We mean that in the usual sense.  I have here an operand, the representation of of a (definite) member of the function's domain, and I want to know the (definite) member of the function's range that the function determines, what we commonly refer to as "the result" of the function.  An algorithm for a function, if we know one, is a (computational) procedure for arriving at the function-determined result given any valid operand.  A coding of the procedure is an expression as data in a form that a computer can follow.

What puts the "functional programming" and "applicative-programming" into Miser and Frugalese is that one kind of computation of a function gives rise to the code for a procedure which can itself be used in the computation of another function, and so on.  This doesn't change the computational idea, it just includes some very powerful and important cases at the heart of the power of digital computers.

Because of the connection between functions and procedures for their calculation, we re-use the mathematical notions as computational ones:

The Frugalese notation

p a 

signifies an apply operation, where

p determines the Ob used as the first operand (the procedure code) and

a determines the Ob taken as the apply's second operand (the data to the procedure).

Apply operations are the Miser embodiment of the stored program principle:

• Every Miser Ob can be used as either first or second operand of an apply.
     
• Programs can treat data as programs and programs as data.
    
• Programs can produce data results that works as programs. 

The sense in which an Ob is being used as program or data is entirely determined by context.  The same Ob may be used one way or the other at any instant.

Note that the apply operation is signified by simple juxtaposition of the term signifying the procedure and the term signifying the data.

What justifies our appropriating the mathematical notion of functional application as a notation for computation?

It is this connection.

If we can show that (Frugalese)  (p a) = (q b) whenever a = b (a and b are the same Obs), then Obs p and q code (not necessarily the same) procedures for the same function: the same function is computed.

We cannot go so far as to say that p = q, because the codes can be different.  Because we are here using "=" for identity of Obs, we would need a different symbol to signify that two obs taken as procedures are procedures for the same function.  There is no such symbol in Frugalese, however.  One motivation of the Miser Project is to sharpen our understanding of why there cannot be such a determination by computational means.

The vice versa case always applies in Miser and Frugalese, of course.  If (as Obs), p = q, then (p a) = (p b) whenever (as Obs) a = b (and a result is determined by whatever p is, taken as code of a Miser procedure).

The Miser apply operation is also called ob.ap. It has been sketched a little in earlier posts.

From now on, we are illustrating Frugalese, not mathematical notation.

4. Streamlining Applicative Notation

Parentheses are used when it is necessary to control grouping into the operands for an apply operation:

p a 

p(a)

(p) a

(p)(a)

all work.

In general,

p q r a

and    p( q( r(a) ) )

express the same applicative operations.  We will often use variations of the streamlined form, p q r a.

These are not equivalent to p q r a:

(p q r) a

(p q)(r a)

[Historical Note: Some applicative systems have p q r a be equivalent to ((p q) r) a.  That is not the case for Frugalese.  If you want that form, you will have to write it that way (or use the alternative form introduced below).  If you run into one of these systems, you will have to be careful when transliterating between that notation, Frugalese, and vice versa.]

5. What About Multiple Parameters?

In Frugalese there are no additional operands in a single apply operation.  In Miser terms, there's one Ob for the procedure and one Ob for the data.  That's it.  One way to introduce multiple-operands of functions is by multiple apply operations:

p(q, r, a) is equivalent to ((p q) r) a

keeping in mind that however p is constructed, it has to be such that we end up with what we have in mind for p(q, r, a).

In general,

f(x₁, x₂, ..., x_n)

= f(x₁) (x₂, ..., x_n)

= f(x₁, x₂) (..., x_n)

= f(x₁, x₂, ...) (x_n)

= (... (f x₁) x₂) ...) x_n.

This particular device was introduced by Alonzo Church, the inventor of the λ-calculus [1].  The basic idea is as follows.

In an applicative expression such as

plus(1, 2)

the 1 and 2 are not two operands of the function plus, the applicative interpretation is

plus(1) 2

where plus(1) is a new procedure that is then applied to to 2 to determine the result. 

If we're very serious about wanting to have a procedure that works on several elements of data at once, rather than deliver intermediate procedures, we can bundle several operand values into a list which is then used as a single data operand in the apply operation:

sum[1, 2, 3]

has the single operand [1, 2, 3], which is a list of the individual operand values 1, 2, and 3. 

The choice between an applicative evaluation such as

plus(plus(1, 2), 3)

and    sum[1, 2, 3]

will depend on the context and whether it is important to single out plus(plus(1,2)), say, as an important intermediate function.  (This is a trivial case meant to illustrate a broader useful practice.)

6. Left-Right, Left-Right, (Cha-cha-cha) Right-Left

In the previous section, there is an additional shorthand that needs to be explained.

p q r a

is    p( q( r(a) ) )

but  p q(r) a

is    p((q r) a) or p q(r, a)

and not p( q( (r)(a))).

The rule is that bracketed elements are data of the procedure immediately to the left, if any.  This is worked out in left-to-right order.  For example,

s p(q)(r) a

= s( ((p q) r) a )

= s p(q, r, a)

= s p(q, r) a

among others.  The choice of particular form depends on the circumstances, the need for parentheses to control essential grouping, and how one might want the features of the applicative expression to stand out.

This rule also applies for list-shaped operands:

s p[x,y](r) a

= s( ((p[x,y]) r) a )

= s p([x,y], r) a

7. Ah, Sugar, Sugar

For Miser, all of these variations are simply syntactic sugar, a sweetening of expression sequence and bracketed groupings that always boils down to a particular combination of apply operations.  The many different ways to group in applicative expressions is a way to express some sense of what the purpose of the expression is.  At some point, there are variants of the notation that I find myself using quite naturally and in preference to conventional functional notation.

The grouping rules are straightforward but it is easy to make mistakes with them.  Small differences can have a big impact.  For example, it is inappropriate to rewrite

s p(q, r) a

as      s (p)(q, r) a

which is the same as s(p, q, r) a instead of s(p(q, r) a).

So long as everything is an Ob in the use of Frugalese notation for Miser, there is very little to help detect unintended combinations.  The beauty of Miser is that everything is an Ob and Obs are interchangeable as data and as procedure (codes).  The difficulty of Miser is that there is no such thing as an inadmissible apply combination. 

This also means that there is no perfect way to pretty-print any Miser applicative expression, although using a pretty-printer transformation into some sort of standard form might reveal an unintended consequence.  That's something worth looking at.

There are more notational features in Frugalese.  The focus here has been on the different ways that apply operations can be expressed for carrying out in a Miser implementation.  The purpose, beside taking a try at explaining it, is to equip ourselves to start putting the notation to good use.

---

[1] Alonzo Church: An Unsolvable Problem of Elementary Number Theory.  American Journal of Mathematics 58 (1936), 345-363.  Reprinted in pp. 88-107 of The Undecidable: Basic Papers on Undecidable Propositions, Unsolvable Problems and Computable Functions, Martin Davis (ed.), Raven Press (New York: 1965), ISBN 0-911216-01-4.

Here Church introduces λ-expressions as part of exploring a bigger question with regard to the unavailability of effective procedures for an infinitely large set of problems.  The original formulation of Church's Thesis is a footnote in this paper.
   Church's tendency was to bracket formulae for functions in {...} and for operands in (...), stating: "A formula {F}(X) may be abbreviated as F(X) in any case where F is or is represented by a single symbol.  A formula {{F}(X)}(Y) may be abbreviated as {F}(X,  Y), or, if F is represented by a single symbol, as F(X, Y).  And ... ."  Here Church is careful to use F, X, Y, etc., as variables for formulas that may be substituted in the given forms.

posted by orcmid at 2/29/2008 07:44:00 PM

SeaFunc: 2008-02-20 Functional Programming Meetup

Who: SeaFunc, the Seattle area Functional Programming interest meet-up

When: 8:00 pm, Wednesday, February 20

Where: Ralph's Grocery Deli
             2035 4th Avenue
             Seattle, WA 98121

We will be meeting in the cafe area against the South wall.  The Seattle web logger meetup will also be congregating at Ralph's, starting at 7:00 pm.  Yes, there is free WiFi.

These are informal gatherings with the discussion following whatever topics the attendees are interested in and fascinated by at the moment.

Anyone interested in the application, extension, or construction of advanced programming languages, functional or not, is encouraged to come.

Although there is a shared interest in functional-programming systems and their languages (Haskell, Scheme, Lisp, ML, OCaml, Scala, F#, ..., <your favorite here>), the attendees also share interest in general advanced programming-language concepts.  Many of these have historical ties to functional programming:

• Interactive development systems and dynamic debuggers
• DSLs: Domain Specific Languages
• Declarative Programming
• Dynamic typing and Dynamic scripting languages
• Generic programming
• Garbage-collection systems and automatic management of dynamic objects
• Parallel programming
• Logic programming
• Program verification and correctness checking systems
• Model-based program development and maintenance

SeaFunc began informal meetings in June 2004.  There is a low-volume Yahoo! Group used primarily for meeting notices and notices about related groups that form from time to time (such as LispSEA, a Lisp-programming interest and advocacy group), interspersed with occasional technical questions and other announcements.  There are no dues or other formalities.  It's all volunteer-driven and self-organizing.

Recently, SeaFunc began alternating meetings in the central Seattle area and the East Side (Bellevue-Redmond) area.   The February 20 meeting is the third fourth gathering of 2008.

Also of Interest:

The initial meeting of a Northwest Functional Programming Interest Group is being held at the Seattle Central Library, 1000 4th Avenue, Seattle, from 5:00 - 6:00 pm, also on February 20.  This get-together is an opportunity to meet others interested in the subject and also ponder the prospects for meetings based around presentations and guest speakers.  A functional-programming interest double-header!

For further information, contact L. G. Meredith at Biosimilarity LLC, +1 206.650.3740.

I'll be at both events to see how we might achieve critical mass together.

posted by orcmid at 2/16/2008 09:13:00 AM