Site Sponsors:
Fast & Easy: Managing SQL Data in Python 3! 
While serving as the Principal Software Development Engineer at Informix, I wrote several articles for BYTE Magazine. Aside from coining the term "DLL Hell" in 1991, I was honoured to work with the same teams that created the Wingz Spreadsheet, as well as what remains the fastest databases on the planet.

Believe it or leave it, for we storage-access enthusiasts times have not changed very much since those days. What has changed has been the massively cool way that we can EASILY add a free SQL Database to our applications.

Database Arsenal: SQL


While I have grown very fond of Microsoft's LINQ, Hibernate, Spring, and even MyBatis, to date nothing comes even close to the quick-and-easy way that we can add the power of SQL to our Pythonic undertakings.

To convince any nay-sayers, submitted for your approval please try out the following 'nifty little demonstration program:

#!/usr/bin/env python3
import sqlite3
""" Log arbitrary "two string" information to an sqlite3 database """


class LogDB:

def __init__(self):
self.bOpen = False

def open(self):
""" Connect to the LOCAL database """
if self.bOpen == True:
return
self.conn = sqlite3.connect('PyLog.sqlt3')
self.curs = self.conn.cursor()
self.bOpen = True

def createTable(self):
""" Create a table for the logged messages """
self.open()
cmd = 'create table logged \
(timestr char(20), message char(256))'
self.curs.execute(cmd)
self.close()

def dropTable(self):
""" Remove the table from the database """
self.open()
cmd = 'drop table logged'
self.curs.execute(cmd)
self.close()

def insertRow(self, timestr, message):
""" Insert an arbitrary logge prefix & message """
self.curs.execute('insert into logged values(?,?)', [timestr, message])

def selectMessages(self):
""" Generator to enumerate thru selected values """
self.curs.execute('select * from logged')
for tstr, msg in self.curs.fetchall():
yield tstr, msg

def close(self):
""" Safe coding is no accident ... """
if self.bOpen:
self.conn.commit()
self.bOpen = False

if __name__ == "__main__":
db = LogDB()
db.createTable()
try:
db.open()
for ss in range(10):
db.insertRow("MyTime" + str(ss), "Message " + str(ss + 1))
for zt, mgs in db.selectMessages():
print(zt, mgs)
finally:
db.close()
db.dropTable()

Designed for more than re-use under SQLite & built around the simple SQL concepts of CREATE TABLE, Insert Data ("row"), Read Row, and DROP TABLE, about the only things lacking in the above is the (1) Update Row (not important in the logging metaphor), as well as (2) a way to unlink() the external database file itself.

The "Pick Two" Dilemma


The above demonstration was also designed for fast & inexpensive data collection activities. In order to satisfy the pedantic, we must therefore surely take a moment to note also that - whenever we plan to manage such entries - for example in relation to another set of database tables - that (3) adding a unique, auto-generated primary key is also recommended.

Come to that, also note how parsing & normalizing data (such as the date & time) might also be something to consider when (4) aggregating these rapidly-collected data into an enterprise schema. (Hence the grand division between "time," "quality" and "cost" for data collection activities in the database world.... but that is another story!)

So while knowing something is great, more than ever before in our modern times have far too many discovered that knowing when to use it (i.e. "pick 2:3") is often quite another story!

It is a "knowledge" (i.e. showing off what we know) versus "wisdom" (showing off how well our product works) type-of thing.


Enjoy the journey,

-Rn




BlogBlock[Python-Advanced-Classes]

[ view entry ] ( 1204 views )   |  permalink  |  related link
Time Formatting & Conversion in Python 3 
For those of yooze-guys whom may not know, Python 3.5 came out last month.

Booya.

To commemorate the event, I thought that I would share a little time-conversion ditty I wrote.

Indeed, the need to understand this particular time-conversion task is so common that I remain amazed that there are - at the time of this writing - absolutely NO decent articles on converting between "military" and "civilian" time on the Internet using Python3.

Short on time as I am, please allow me to post and go - for the moment we'll let 'google do the rest:


from datetime import datetime as zdatetime
from datetime import timezone

class LogEntry():

""" Manage a GMT (i.e. Non-naive timezone) log-entry """
def _hack(self, message):
znow = zdatetime.now(timezone.utc)
self.utc_date = znow.strftime(format=self.UFORMAT)
self.message = str(message)

def __init__(self, message):
""" Taking care of beez-nice """
self.UFORMAT = '%Y-%m-%d %H:%M:%S [%z]'
self.LFORMAT = '%Y-%m-%d %I:%M:%S %p'
self._hack(message)

def __repr__(self):
""" Get the UTC-0 (i.e. non-naive + military-time) format """
return self.utc_date + "\t" + self.message

def __str__(self):
""" Get the LOCAL (naive + civilian-time) format """
st_time = zdatetime.strptime(self.utc_date,self.UFORMAT)
ltime = st_time.astimezone()
return "[" + \
ltime.strftime(format=self.LFORMAT) + "] " \
+ self.message

def hack(self, message):
""" Update the time, as well as the message """
self._hack(message)

if __name__ == '__main__':
""" Basic proof of concept """
ent = LogEntry('This is a test')
print(repr(ent))
print(str(ent))

Here is the proof of concept:

>>> ================================ RESTART ================================
>>>
2015-10-25 00:04:12 [+0000] This is a test
[2015-10-24 08:04:12 PM] This is a test
>>>

Enjoy,

-Rn

p.s: Just for the record, when logging / storing events of any worth onto servers, the anticipated & common convention remains to store said textual line-entries in a GMT / UTC-0 time format.




BlogBlock[Python-Advanced-Classes]


[ view entry ] ( 1150 views )   |  permalink
Pay-As-You-Go -v- Global Initialization 
While easy to understand in Java, the opportunity to initialize data & classes prior to their use is a well known technique.

Here is a Java:


static ColorPair[] colors = null;

static {
List<ColorPair> array = new ArrayList<ColorPair>();
Color[] ccc = {
Color.BLACK,
Color.BLUE,
Color.CYAN,
Color.DARK_GRAY,
Color.GRAY,
Color.GREEN,
Color.LIGHT_GRAY,
Color.MAGENTA,
Color.ORANGE,
Color.PINK,
Color.RED,
Color.WHITE,
Color.YELLOW
};
for (Color a : ccc) {
for (Color b : ccc) {
if (a == b) {
continue;
}
array.add(new ColorPair(a, b));
}
}
colors = new ColorPair[array.size()];
array.toArray(colors);
}


Indeed, when we are guaranteed to use static content, then taking the time to pre-initialize objects is a common practice. Why would anyone want to do anything else?

Yet when calculating the total latency in pre-execution load times, on more than one occasion I have been found re-factoring static initializers so as to allow programs to be more responsive.

Pay As-You Go


Given that the above example has been defined as package-protected, surely it can be declared private.


private static ColorPair[] colors = null;


After so hiding the data structure, the very next thing to do might be to expose the structure with a more visible member function.


public static ColorPair[] GetDefaultColors() {
return colors;
}


Once so exposed, a far less costly initialization strategy can be created. -Specifically, rather than requiring colors to be pre-initialized, we can now pay-as-we-go by only allowing the initialization loop to run when the data are needed:


private static void init() {
List<ColorPair> array = new ArrayList<ColorPair>();
Color[] ccc = {
Color.BLACK,
Color.BLUE,
Color.CYAN,
Color.DARK_GRAY,
Color.GRAY,
Color.GREEN,
Color.LIGHT_GRAY,
Color.MAGENTA,
Color.ORANGE,
Color.PINK,
Color.RED,
Color.WHITE,
Color.YELLOW
};
for (Color a : ccc) {
for (Color b : ccc) {
if (a == b) {
continue;
}
array.add(new ColorPair(a, b));
}
}
colors = new ColorPair[array.size()];
array.toArray(colors);
}


Note in the above that the initialization still takes place as before, but not every time our program is loaded. Rather, we can "pay as we go" by allowing the structure to be initialized only if, as well as when, the data are required:


public static ColorPair[] GetDefaultColors() {
if(colors == null)
init();
return colors;
}


Conclusion


Whenever I find myself migrating a bit of code from a project into a support library the first thing I look for is an opportunity to convert any global / static initializers into a pay-as-you-go strategy. Of course, given that one of the first principles of good Object Oriented Design is Encapsulation, from a puritanical point of view many often find very little justification for doing anything else.

Enjoy the Journey!

-- Rn

[ view entry ] ( 1214 views )   |  permalink

<<First <Back | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | Next> Last>>