5 Replies Latest reply on Nov 21, 2016 9:00 AM by robinsc

    Plenty Long time database managers.

    john f kennedy

      This query comes from an old database manager from the 80's when multidimensional, relation databases were new products.


      What happened to computing?  In the early 80's DOS-based PC's were just starting to hook up to mainframes and the mini computer revolution had come and gone.  New software and incredibly fast hardware in new IBM PC's was advancing the knowledge base of people, particularly in businesses, at phenomenal rates using unprecedented amounts of data, but also saving lots of time.


      My phone now has significantly more processing power than any of those IBM PC's and probably more than the mainframes, yet so much of the world's hardware/software combinations are so slow and complex, productivity is going backwards.  What happened?


      I remember a job I was given at a large company in the early to mid eighties that had tens of thousands of customers, hundreds of sales reps, hundreds of products and numerous States.  My job was to produce a monthly report for the Board of Directors using information from Mainframes, portable sales systems on trucks & vans uploaded to depots, data from hand held POS devices from Reps, Apple and Mac computers from the Research & Development guys etc etc.  I was told it would take 28 days each month to the complete the report,and it did.  consequently, Board meetings always discussed 'last months' data.  I thought this ridiculous and life was way too short to spend every day of every month gathering data from all of these disparate systems that never talked to one another.  I would stay up all night running execution files on terminals all over the building, then getting floppy disks from the scientists one week, from the State offices the next etc etc.  Crazy.  I then came across a new DOS based PC software package called pcEXPRESS, owned by Information Resources (later known as IRI) but later purchased by Oracle, apparently morphing into "Personal Express".  It was a database and it summarised data - exactly what I had to do for 28 days every month - only this could produce the reports within hours - and it did.   The problems was, it still took me 25 days or so to key in all of the relevant data from all these disparate systems all over the company, located all over the country!

      Fortunately, persistence (and plenty of sledge hammers!) paid off and we able to connect all of these disparate systems physically - DOS-based PC's with Mainframes with Apple PC's etc and within a few months, the database would be updated each month in a matter of hours.  All I had to do was chase the humans down the line to make sure they did their jobs on time and made sure the data was both timely and accurate.

      Before you knew it, Board reports took half a day, not 28, which left me half a day to drink cocktails on the harbour and another 27 days to work out how we could use multidimensional, relational databases to generate a huge competitive advantage.   And we did. The intelligence we garnered was incredible.  I was promoted, given an almost unlimited expense account, staff, travelled the world attending user conferences and a whole new way of life.


      30 odd years later I wonder what happened to computing?    I now own and run a small business and have to operate in a world of never-ending and ever-changing compliance regulations - accounting, payroll, superannuation, sales reporting, taxation, the list goes on.   There is no time left in an average day and so I look toward technology thinking "what happened?"   I would love help with our payroll software; it's dumb.  Our accounting software is laborious at best and outright frustrating when things are going well.  What happened to technology?   Why am I still working 13 hours a day, seven days a week 30 years later????


      I guess this is a long winded background, so my question to the old guys and gals on this forum is "What happened to pcEXPRESS  and it's offspring?   It was simple, efficient logic that appears to have suffered the same demise of other pre-Windows packages like Harvard graphic, Lotus 123 and Wordperfect.   The never ending pursuit of GUI and the stifling dominance of MS might have also killed off simple databases?.   Anyone know much about this?


      Is Personal Express still really simple to use, even for an old Business Intelligence dude like me?  Do I still have time to go out and save the world and still  enjoy cocktails on a beach by 3.30pm?



      If you are unsure of how pcEXPRESS worked, it had simple commands to mine the data - using it's relational structure to save lots of time and it's multi-dimensional nature to expand or contract as required.


      By way of example, lets imagine a company with:-

      A. 260 products - then classified into 8 categories (eg: potato chips, corns chips etc), 6 Manufacturing facilities, but also into 12 brands (already a 4 dimensional array of data).

      B. Days of the week - then totaled by week (7), month (28-31), year (354-5)etc

      C: 35,000 Customers - then classified into Rep area, manager area, State, Customer Type (Supermarket, Route or Vending), town, sales region, etc.


      I've probably long forgotten other "dimensions" we used, but it quickly became a large database.  The beauty of pcEXPRESS was it's simplicity and ability to calculate quickly based on the mining of the data.  For example, if the database contains sales information for 3 years but you want yesterdays sales total only, it only has to calculate on a mere fraction of the database (1 divide by 365 x 3).  If I wanted to know the sales from a particular Rep for a particular product sold thru a particular retail group over a certain period (eg: he was conducting a trade promotion with that customer group) it was pretty easy.

      The (one off) query would have the following syntax:


      Limit Region to Victoria West

      Limit Rep to michael jones

      Limit Category to Corn Chips

      Limit month to April to june

      limit year to 2012


      This simplicity wold be perfect for voice controlled data queries.




      Results were almost immediate.  Nothing complicated.  


      If I made a "limit" command, the database query was immediately reduced to that size of that dimension, so if I said "limit category to Corn Chips", everything I did thereafter only concerned corn chips,  If I typed "total sales" it would add every single corn chip sale and give me the dollar total.  If I typed "total units" it would report the number of corner chip products sold.  If I wanted to clear all limits and return to the entire database, I typed "limit data to all."  It was pretty easy, really quick and very reliable.  There was no complicated syntax, nor hard to remember 'controls' like "<" or "&" etc.  It was all English.


      Nowadays, databases are used all over the place but they all seem to be so "dumb"!   I use Accounting software and it seems to take forever to load,  When I am using the Payroll section, I select an employee but instead of limiting pay rates only to the award applicable to that employee, it shows everything; every employee's possible pay rate, in a time consuming drop-down box  Why???.  Ugghh, slow, time consuming.   Ditto if I want to key in a sales receipt - once I select a product type, I should only ever see the tax rates applicable to that type of product, but no, you have to choose from a long list every time.   Ditto this, ditto that.  There are no relationships set up - all software seems really dumb now, making life more complex and far more time consuming than it should be. 


      Computing now encroaches on almost every hour of our lives and for some people it's much more than that.   Watches are now computers, so too phones, fridges, cars and pretty much everything else in between.   Computing is very "pretty" now - try to read a news article and a video is forced down your throat whether you want it or not, graphics are incredible, but for some reason, the programs that were meant to give us shorter working days have lost the plot.  What happened?


      Are databases still nice and quick and simple to use like they once were?



        • 1. Re: Plenty Long time database managers.
          Mike Kutz

          Wrong place for a rant.


          Place in: DBA/DEV Community Corner

          Or make a better argument for your "new to oracle" syntax here: Database Ideas



          databases aren't dumb.  The user interface is.


          New data language suited for Voice Command would be neat.

          However, it will need to be "more natural" then what you posted.



          "Computer:  how many people, by rank, are located in 10 Forward?  give them in order by top rank first" -- Picard


          The database can already handle that.

          It would be up to the voice engine to convert this to an appropriate SQL statement.


          select  e.job, count(*) N

          from current_people_location p

          join emp e on p.empno = e.empno

          join jobs j on e.job=j.job

          where p.location_name='10 Forward'

          group by

          e.job, j.job_rank

          order by j.job_rank desc



          • 2. Re: Plenty Long time database managers.
            Paul M.

            for some reason, the programs that were meant to give us shorter working days have lost the plot. What happened?

            I can only talk about my personal experience, and I think that there are too many bad programmers around the world.

            Many years ago (in the seventies) a colleague of mine wrote some programming language manuals (Cobol, Basic,....), and in their preface he used to say "programmers tend to consider themselves geniuses" : to demonstrate it, they write very complicated codes.

            Unfortunately it's the exact opposite : the genius is the one that does simple things, which work better, faster, and are easier to be maintained.


            Always, in my experience, when a solution looked too complicated, surely there was a simpler one (hence better).

            • 3. Re: Plenty Long time database managers.
              William Robertson

              Are databases still nice and quick and simple to use like they once were?

              I think databases are fine and a standard Kimball-style reporting datamart would give you exactly what you want. Unfortunately the OOP crowd has taken over the asylum to the extent that nobody in big corporate sites is interested in building those. At my current project they call it a "cache" and have a separate team using Apache Spark for the Projection Layer as input to the Universal Data Model application which will generate the final report. I had to explain to my BA what a Logical Model was, and I'm still not quite sure he gets it. They have meetings about long lists of "attributes" and keep asking what layer the aggregation will go in. It's starting to look as though Data Management professionals no longer know how to manage data.

              • 4. Re: Plenty Long time database managers.

                John F Kennedy wrote:


                Are databases still nice and quick and simple to use like they once were?


                They are  better.


                I remember DOS-based database products, like PC Express and others. My first "client-server" system was a Clipper compiled dBase3+ application, with the enormous 4MB database residing on a 20MB hard drive of a Novel file server. The largest table had about 2000 rows or so. The database had 2 to 3 users.


                Today, my largest database is over 10TB and largest table has several billion rows - with over 40,000 rows being inserted every second. I have over 500 corporates (from government departments, retail and wholesale companies, to banks) using it.


                And that is what happened to databases. Faster. Larger. Better.


                What did not grow equally with database theory and technology, is the human side. There is in my view, widespread ignorance in today's IT environments about database concepts and fundamentals. Do not even get me started on the levels of utter ignorance about databases, amongst Java developers... <sigh>


                All of which results in many a poorly designed system of today, with all kinds of "database issues".


                But despite this level of ignorance, database technology is one of the primary driving factors of the Information Age.

                • 5. Re: Plenty Long time database managers.

                  Also the horrors of btrieve where any table can contain unrelated records ! basically worse than even clustered tables. the past always looks better cause we see it through rose tinted glasses.