Progress Report 4: Java

Yesterday we cleaned the home. We didn't just clean it. We made it more cleanable. I love that. One thing that seemed pretty clear to me by the end is that a project completed is worth more than a few projects merely begun. Despite that fact I still kind of want to start the student consulting thing. But for right now there is plenty to do.

I am a little bogged down in Java right now. I think that by the time the program is written I will have to be a Tenderfoot. Right now I am not yet a Scout, much less a Tenderfoot. But I will say that I love it. I think it is powerful, rich, and beautiful. I love the way the structure works. I love the accessibility and the universality. I'd like to find excuses to use the language in my numerical simulations down the road and also in making presentations.

As far as the signal processing side of things I am feeling pretty good now. I found a great text, the computer music tutorial, by Curtis Roads. Then using key words from the text I found referrences to many modern methods for solving the "pitch detection problem." The methods that I came up with on my own were there. But there are many others as well. So the math part I now feel pretty good about. It's just the real time control of sound data using Java that we really need to get a handle on. It's a little iffy whether we will make the deadline or no. It still seems possible. But we'll see.

It sounds like Denis is making some good progress. We'll get there soon enough!
Doug

Progress Report 3: Arches

A good chunk of the week was spent either preparing for or going to or being at Arches National Park. And what a treat it was! Seeing the kids climbing on the redrock, digging in the sand, exploring the brush and learning about the animals and plants gave me a feeling of contentment as a parent that I don't always have. (I want so badly for the kids to see and do some of the neat things that I have been able to see and do!) What made it better was to have family with us. Gina's mom came and was delightful with the kids. And then Peter and Jonathan came for the tale end. It was great.

So progress may have been curtailed by all the activity and absence. Still, I found and downloaded NetBeans and NetBeans IDE + JDK 5.0 Update 6 and installed them. Our sound card wasn't recognized by Windows so I had to find it and reinstall it. And we bought a $10 computer microphone from Walmart so now I can input sound. (Google Talk anyone?) I wrote my first applet and applications in Java using NetBeans and compiled and ran them successfully. I learned briefly what object oriented programming is. (Yes, I really am that much of a novice with regard to computer programming. I had to learn a little before I understood what I was doing with NetBeans.) And I complemented my simple Fourier domain algorithm for pitch determination with another simple algorithm that works completely in the time domain. They are both my own algorithms, which is probably bad. I should search the literature some more to see what others have done.

Anyway, now the tools are in place. I can input audio, I have Java, an IDE, and all of the necessary documentation. I think the next step is to start playing with sound. The goal for this week, then, is to start manipulating real time sound. Once I do that I want to display real time and fourier transform graphs of the sound patterns coming from my voice while I sing a given pitch. Then it is time to begin perfecting the pitch determination algorithm. So it will be time to get to work. Tomorrow.

Progress Report 2: running

When I was in kindergarten we took a bus home from school that dropped us off at the bottom of a large hill. I had lots of energy so often I liked to run home. I didn't run the hill all at once though. Instead I picked chunks. I would look at a car in the distance and say “I'm going to run until I get past that car.” And I'd do it. Then I'd pick the next object. It felt neat to get to each object. It felt like I had accomplished something. This week I expressed to Gina some of my doubts about the direction that Denis and I are taking and whether we would succeed in the end. She counseled me to stop worrying so much about things far far away and make very concrete goals and accomplish them. Smart lady. The thing is that if we take this route the project will be rewarding no matter what. But I think that it will succeed. Right now there are not very many routes to failure. We can't go bankrupt because we have no expenses. Pretty much the only way we can fail is by quitting. And that comes from either discouragement, distraction, or boredom. And those come from either forgetting to look at the big picture or forgetting to go for and then appreciate the small successes along the way. This blog is meant to help me avoid both of those pitfalls.

Now I still take the bus for school. (I guess some of us grow up slowly!) I take the bus to campus and then walk across campus and finally across a major street called Lomas to get to the UNM physics building. The stop light for Lomas is lopsided. It is green for a long long time for the traffic (mostly cars) that goes east and west. And then it is green for a short time for the traffic (a lot of pedestrians as well as some buses and cars) that goes north and south. So there is a short window of opportunity and a bona fide penalty if you miss it. Anyway, when I learned this I began to run when I saw the light turn green even if I was a fair ways from the intersection. I would sometimes make it across in the nick of time. This week I had stayed a little too long on the south side of campus helping students solve physics problems and I was going to be late for the information physics group meeting. A new thought occurred to me: why not jog while the light was red? Then when it turned green I would be there ready. So I did. I think that is a pretty significant change in thinking. In the beginning I didn't recognize that a green light was an opportunity. So I didn't really notice it. Later I recognized it and got up the energy to go for it. Finally, just recently I began anticipating the opportunity in advance and getting myself into the perfect position to take advantage of it. Anyway, what I've learned is that if you want something there may be an opportunity that will come up. Your job is to move now while it isn't there so that when it does come you are ready.

I don't know that this business was the best choice of the 100 and some odd. But in a way it doesn't matter. This business is a way to begin running, a way to move forward so that as the future arrives we are prepared. Have a good week. Doug

Progress Report 2, business

I have a couple of doubts about the way we are going about this from a business standpoint. The first is that the more I read about successful technical businesses (IBM, HP, or even Thomas Edison's companies) the more I believe that they probably need to be sales driven. Sales and marketing. They need to be about serving people or meeting their needs or wants somehow. So I am nervous to be investing in technology first rather than in people first.

My second doubt is whether we should be the ones to do the hard core technical work. Why should I be programming when there are literally millions of people out there who are all more computer literate than I am and probably tens (hundreds?) of thousands of people who are both more computer literate and have a better understanding of the whole acoustics, digital audio world? From a business standpoint the answer is that I probably shouldn't. I should find people who are much more qualified and have them come along.

So why are we starting like we are starting? For a few reasons, I guess. First, because the stuff is so cool! At least at the beginning I think that we want to have the technical fun. In our world the fact that we don't know things is exciting. It means a steep learning curve and we love that. Second, because it will be nice to have at least some understanding of what is going on when we do have to bring on some experts. And it will be easier to sell the experts of the ideas if we have already done some of the work. Third, because we can afford to. We are both solvent without this business. We haven't given up our day jobs. So even if this is the slow way to do things we can go ahead and take this route for the fun of it.

Finally, I think that the learning is important for another reason as well. Better ideas come when you have more specific knowledge. And you can better judge the soundness of those ideas. I'd like to think that even though this may not be the quickest route toward building something up it may still be a good one. Still, marketing and technical talent need to come into the picture at some point not too far from now . . .

Progress Report 2, technical

It's been a week and a half and we are now in that phase of the project where the project begins to expand before your eyes. Quick recognition of pitch is seeming more complicated than I originally thought: the lower frequencies that people normally sing at are in the 50 Hz range. A simple Fourier Transform rule says that at that speed the best you might expect to resolve the frequency in 0.1 seconds is on the order of 10 Hz, which is a spread of at least a half note on either side. For a voice with the right timbre (harmonic, and it looks like most Western musical instruments are more or less harmonic in timbre) I guess that you can use the spacing of the partials (I think partials is the word they use for non-fundamental frequencies). In addition it is looking like a simple Fast Fourier Transform may not be the best way to go and also like a typical person's voice may have a fairly broad spread around the fundamental frequency in the first place. I could go on but you get the picture that there is still a little learning to do in terms of acoustics, signal processing, and human perception of pitch.

On the more technical side of programming things are moving forward. It looks like there are a ton of resources out there to help you begin processing digital audio with Port Audio being just one. Sun has a very well documented sound API in Java so we are leaning toward Java. We've heard it said that Java applications can be slow, but the proof is in the pudding for our application. I figure that once we have something written it shouldn't be too hard to take to another language and compiler should that prove necessary.

This is super interesting and fun from a robotics standpoint as well as from an applications standpoint. I'm starting to realize that computers now have quite a few input and output ports that could be used for a lot of different things. How would you like to control a remote robot using your sound card? Or a USB port? Or your Ethernet port? Or look at the huge amount of information that is communicated to a computer's monitor. Right now in the lab you have to buy special cards if you want to have analogue and digital input and output ports to control devices like the microscope that I was adapting. I think it could all be done through the ports that the computer already has. And with a proper adapter you could get all kinds of signals into and out of your computer very quickly and simultaneously. I think that giving the computer senses (good sources of local information) and hands (good ways to get things done) is a neat concept. The cool thing is that nothing need be changed about the standard pc to add these extra functionalities. It would only take a little external hardware and some software. And it doesn't take too much imagination to see how all of this can work toward inexpensive medical devices . . . the goal would be to have a million devices that all work off of one adapter. Just change the device like you change out one color of lego block for another.

Progress Report 1, lessons from kids

So I figure the weekend is a good time to give progress reports and write thoughts. You can expect something each Saturday or Sunday.

I think that getting a working prototype that will be worth selling completed by April 15 is probably the most serious technical challenge I have faced. I'm not a programmer. I don't know how to make a windows program without using expensive proprietary software tools. I don't know much about digital audio technology. I just understand the physics of sound and have a really cool really simple program that I want to write.

Here is what I have learned so far:

• It's nice to have a partner. Denis (V. Seletskiy) is just as excited about all of this as I am and perhaps more capable.
• Sound is transduced by the microphone from vibration into voltage oscillations. The sound card converts those oscillations into a digital signal by sampling them at some rate (the rate ought to be faster than twice the highest frequency you care about by Shannon's theorem and the highest frequency we can hear is about 20kHz so 40kHz and above is fine) with each sample just changing a voltage amplitude into a digital number. The digital numbers are typically 8 or 16 bits in length and I think that you may be able to choose the format.
• There is a nice audio programming library (http://www.portaudio.com/) available for free for real time manipulation of sound. They make no licensing restrictions on products compiled with their library so I am more than grateful.
• I still am not sure what happens to the sampled data while it is being taken or how the input and output functions work. I don't know if certain file types are generated automatically or if it is put into some kind of buffer or what. And I don't know how to find out.
• I don't know anything about programming for windows. If you know of any resources that would help me with either side, that is with the user interface or with the real time manipulation of sound, please let me know. I want to make that deadline.

Lessons from kids.

This first one is from when my little brother Andrew was about 3 or 4. We wanted to do something and he needed to finish his food first so I asked him to hurry. "Okay!" he said. So he took a bite and chewed frantically and started running around the table. Then he got another bite and ran around the table more. He felt like he was hurrying. But he ended up going slower than before. I thought it was hilarious but I also tried to remember it so that I could notice if I ever did the same thing.

The second one is from this morning. Kate didn't want to help us clean the house. Actually she wanted to but didn't want to be cold so she couldn't leave her blanket. I ran to get her her coat but she couldn't get it on because it was too small. Then she couldn't clean because Abby made Kate's bed but not how Kate wanted it. But she couldn't redo it because taking apart the bed would be like messing up the room and we were trying to clean. So I pulled off the covers for her so she could make her bed. Then she hid under the sheet and couldn't make it because she was stuck. I got her unstuck and . . . you get the idea. The thing is I think she half believed her reasons. And she was miserable the whole time. I sat her down and taught her the concept of lying to herself about what she could do by telling the story of her morning. Then she started working and stopped finding reasons why she couldn't and she was remarkably cheerful. Happy. Gay. And productive. So the question is "How much do I do that to myself?"

Until next week,

Doug

Choice Made, blogs more rare, a goal.

So the choice is made at least for now: musician feedback and music DDR. Feel free to use any ideas posted here.

Anyway, there is competition. We've scoped it some. But we like competition because it will force us to improve ourselves. I think we can provide something of value.

Anyway, now the blogs will become more rare. I will try to make an update once a week. But really I only have about an hour a day that I can spare for this whole venture so I'm going to be devoting that to action.

But let me state right now the first goal: a distributable working prototype of a simple system available by April 15th.

Doug