Electrified Road Trip Part 2: Tracking with Teslamate

In preparation for my big road trip, I added Teslamate to my Home Assistant setup.  I have to say, I am sad that I didn't do this on day 1.  This little platform is amazing, and it not only gets a huge amount of data out of Tesla's cloud and on to my own hardware, but it also generates some wonderful Grafana dashboards too!  It is kind of a game changer compared to the way I have collected stats from my current car, and just using the unofficial Tesla Integration for home assistant.

So, the integration's main screen doesn't look like anything special.  The map shows the real time location of the car, along with some basic stats about the car.  You can also use Geo-fences to give common locations any name you like, and you can even set a charging price for the location.  This is particularly useful for charging at home because there is no receipt for purchasing power like you get from a charging station.  I will get into manually updating charge prices once I figure that out, but I rarely charge anywhere other than home, so it is a low priority task for me to sort out.

The real advantage comes with 2 backend features this plugin provides.  First, the plugin integrates with a database to log all your data about your car.  This data goes far beyond just what you see on the plugin page, and this is why Grafana is the real hero for this plugin.  The second feature is MQTT, which is just a protocol used by IoT devices to communicate updates.  Once you have these two features, you can create dashboards in Grafana to visualize the data and create reports, and MQTT sends the data to home assistant so I can trigger events based on the reported stats.

To the right I show MQTT explorer, which shows an example of the message data Teslamate sends to Home Assistant.  This data gets updated in real time, and one thing this does for me is update any time the car is unplugged from the charger immediately, rather than having to wait for the integration to poll Tesla for the data, which can take anywhere from no time to 5 minutes.  One problem this caused me a few times is that I would unplug my car, and have Home Assistant log the data to a google sheet; however, if the data didn't get logged immediately my car would not be at home when the data was logged, and my cost estimates would be off.  I guess Teslamate removes the need for this Google Integration too, which I appreciate reducing the number of cloud dependencies I have.

Possibly the best part of the this addon is Grafana Dashboards!  For those who don't know, Grafana is a web application that can collect data from databases and organize it into Dashboards, which are a collection of charts, graphs, and data.  It is somewhat popular for its ability to collect data from many places and combine it all together into a single screen/report that is full of pretty pictures.

Teslamate has a bunch of dashboard that is loads into Grafana if you set it up correctly.  Its a great way to visualize things like your most recent trip data and commute data.  For example, this image shows a track for everywhere I've driven this month. When I scroll down through the list of drives I've made, I can tell that I drive the same few routes over and over again, and it gives some nice summary information like how efficiently I drive (256 Wh/mi) and the fact I have paid about $13 to drive my car 265 miles (which is WAY cheaper than gas).  Just getting access to this data is a critical asset to accurately estimate my personal cost of driving the car, and having a few months of data under my belt will give me a clear picture of how far I can drive my car on a single charge.

For those interested in the process of getting this all set up, I added 3 home assistant add-ons using the instructions on the mat-FFFFFF github repository here.  If you are savvy with docker and have a spare raspberry pi laying around, I have to admit most of the troubleshooting resources online focus on putting everything into its own container.  However, its not too difficult to follow the directions and get it all setup, but maybe that could be a good thing to write about next.  Let me circle back around to that once I get around to refactoring a couple things for best practices.

Anyway, that's it for now.  Until next time.

Later,

SteveO

Tesla Purchasing Experience

 So, as I'm getting used to having a Tesla, I thought this would be a good time to look back and reflect on the purchasing experience.  This is a really unique experience, and I think it is much better than most of the horror stories I read about on social media.  So, for reference, lets go back a couple of months, and walk through our car buying experience this time around.

This spring, my 2007 Honda Accord really started to show its age.  The windows stopped working (again), the AC stopped working (again), and it started feeling a little more sluggish than before; basically, it needed some TLC, and I had no more love for it.  So we decided we were going to get a new car; we wanted an electric car.

Day 1, I decided I was not going to get a Tesla.  I think they are too proprietary, and there is no excuse not to have Android Auto and CarPlay built in.  I still feel this way, and the entertainment system just feel crappy in general while driving.  We have YouTube Music, and that comes with Ad free Youtube, so we won't consider the supported streaming services. While in park, I can use abettertheater.com to play movies/shows on Plex, and then we have YouTube, Netflix, and Disney+ in the theater, which is fine.  Games are okay, but we all have switch game consoles, so we don't care.

Anyway, So, we voyaged out to the dealerships.  We went to Nissan, Kia, Hyundai, Subaru, Ford, Chevy... we just spent a couple months looking up every electric vehicle we could find.  For reference, every single one of them had Android Auto and CarPlay standard - its just expected in every car (yes, I an REALLY hung up on this).  All of them also (mostly) felt like normal gas cars (ICE is the hip term for this BTW, for Internal Combustion Engine).  At least, they had a mode that felt like a normal car, and they would have a branded name for a single pedal mode that applies regenerative breaking when you take your foot off of the gas.

I think what finally swayed me to consider a Tesla was how crappy the experience of going to a dealership is.  You go in, they show you the top end model even though you say you were a little weary of paying the base model price for a car.  They show you a car or two, and they you sit down in the room where they try to pressure you into getting the car or pre-ordering.  Honestly, we wanted a Chevy Bolt EUV.  It would meet our needs and it was by far the cheapest car of the bunch, but because GM is stopping production in October, we just couldn't get one.

The next couple options we were looking at were the Kia offerings; the Nero or the Hyundai Ionic 5.  The biggest issue here is we wanted the seat memory, which was only offered on the top end model.  VW and Volvo also had some compelling options, and the Ford Mustang was a strong contender, but going to these dealerships was starting to wear on me and leave a bad taste in my mouth.

One day, I was talking to someone at our daughter's school, and they just purchased a Model Y.  It turns out I wasn't paying attention that the price dropped substantially on the Model Y; also, the base model car has pretty much all the features on it.  The other trim models only changed the battery and added the option for a 3rd row of seats (we didn't really care about the other differences.)  So, I decided to try and test drive a Tesla Model Y.  This is where the story actually begins.

I go to the Tesla website to start sorting out models, trims, and other options and find 3 somewhat straight forward options.  I see a big link to schedule a test drive; however, I click a referral link that I got from my friend.  All Tesla owners get a referral link, and the person who refers you get points that can be redeemed for stuff.  For example, the person who referred me got 3 months of full self driving because I bought a car with his link, and some other points for the test drive.  I got $1000 off my purchase.  I scheduled a test drive, uploaded a photo of my drivers license, and signed a waver.  They sent me a calendar invite and a few links to YouTube videos with instructions about how to navigate the touchscreen UI and start the car.

A few days later, I show up to test drive the car.  They have the cars all lined up in front of the dealership, I walk up to some and they take me to the front desk.  It takes them a minute to find the right key card, but a quick peek at my insurance card, a signature on a temporary registration card, and we are escorted to a parked car in front of the building.  There were a couple questions and comments about starting and how different the car is to drive. However, within maybe 10 minutes of showing up I am driving the car I am considering to buy.

At this point I am used to driving an electric car that uses the gas pedal to trigger regenerative braking, but the Tesla has a very aggressive regenerative brake.  I kind of slammed on the brakes a couple times during the drive because I didn't realize I lifted my foot off the pedal.  Then, I got on the expressway, and this is the moment the Tesla kind of stole my heart; and put a big smile on my face.  Its just a lot of fun to have a car that goes when you push the gas pedal.  We got back and the lady at the dealership asked us if we had any questions, we had a short conversation and went home.  Later that night, my wife picked on my a little because I played Forza after test driving the Tesla.

It was the best experience I had ever had test driving a car.  No pressuring me to walk out the door with a contract, and no trying to upsell me on stuff I didn't ask for.  I loved the car, even despite the crappy music situation.

We went home and talked about it.  We made notes and compared the different cars we drove.  The only thing that would have stopped us from getting a Tesla was if the Bolt were available.  It had all the features we wanted for $20k less, and we primarily wanted this as a commuter car, so we wanted a smaller monthly payment .

However, in the end, we decided because the Bold wasn't available, the Model Y was the next best option.  Most of the other cars were too short, which meant I would hit my head getting in and out.  Also, in order to get seats with programmable memory, every car except the tesla required us to upgrade to one of the premium trims, which were usually $10-20K more than the Tesla.  The Mustang almost shined here, but we wanted all wheel drive for taking our daughter to Ski Class this winter with her school.

One day, sitting in Chick-fil-A I installed the Tesla App on my phone, chose the options and customizations on a Model Y, and tapped buy.  From there, it was a matter of adding my wife and I as owners of the car.  We signed up for financing through Tesla because they had the best rate we could find, and even initiated the trade in estimate.  The whole process took place on the phone.  Then I got the request to schedule delivery; I chose a date and time that worked well for me, and we went to go pick up the car.

In the mean time, I watched countless YouTube videos about checklists that people made to inspect their Tesla when they accept delivery.  I felt well prepared to inspect every corner of the car when I arrived.  We drove our old car to the dealership, and went inside.  Someone was standing at a podium, I said we were here to pick up our Model Y, and he said you walked right past it.  He handed me an envelope with some documents and the key cards to open up, and walked me to the car.  He told me to click a button in the app, stood there for a minute to see if I had any questions, and then told us to come back in to see him once we looked over the car and were ready to sign.

We walked around the car, opened the trunk and the frunk.  We sat in every seat, ran our fingers along all the trim, paired our phones to the Bluetooth, Made sure it locked when we walked away and unlocked for both of us.  We were happy, so we went back in to finish signing at a table and hand over the keys to the old car.  Again, compared to the other cars I have bought, this was less paperwork, no pressure, and just straight to business.

A couple days later, we found a couple issues.  One of the exterior cameras had a chip in the enclosure, the panel over the seatbelt wasn't aligned correctly, and one of the buttons on the steering wheel had an issue when pressing to the left.  I scheduled a service appointment through the app.  One day, I guy from tesla pulled into my driveway, replaced the defective parts, aligned the panel, and said it was all covered under warranty.  We also ordered the garage door opener, which I didn't realize was extra.

All said and done, it was painless, getting the issues serviced was easy (especially because I work from home).  Everything was pretty quick and simple.  As I've said a couple of times, this was the best purchasing experience I've ever had.  No sales people trying to get me to buy today, no dropping my car off for a 10 minute repair.  It seems like all the horror stories of delivery didn't happen for me.  

I still really hate the entertainment situation.  Spotify worked just long enough for me to try and sign up for a free account.  Then I got hit with the "premium account required" message.  I didn't like the experience on Spotify, so I'm not buying it.  I then signed up for Tidal; I even paid the $2 to get a 60 free trial; its better than Spotify, but I really wish I could just use YouTube Music on the screen.  Even the Bluetooth experience is lacking compared to other cars.  My phone won't play music automatically when it connects like pretty much every car I've use Bluetooth with.  I also can't browse playlists on my phone through Bluetooth, which is an option with plex amp.  On Tesla, Bluetooth only allows play/pause, next, and previous.

So, I still hate having a locked down car, but at least the rest of the driving experience is good.  I also think the video player will come in handy when I go to a supercharger.  I also find it very strange that the Tesla doesn't create a WiFi network, so if I purchase premium connectivity, the kids can't use an iPad to watch movies in the back seat while driving.  I guess its not the end of the world so long as we download some movies/shows before leaving the house.  Anyway, I'll get more into that as we get closer to our road trip.

Later,

SteveO

Electrified Road Trip Part 1: A tale of two maps

 

So, we recently became the owners of a new Tesla Model Y.  We went with the base model, and only paid to change the paint color and add a trailer hitch (mostly for a bike rack.). I have actually plotted this trip twice so far.  Once with the Tesla App, and again with abetterrouteplanner.com (ABRP for short)

I have to admit, I like ABRP more, so it seems to live up to its name.  The nice thing about both tools, is they break the route into legs, where you need to stop at a charger.  They also both had some functionality for estimating charge times, and how much to charge the battery to make it through the next leg of the trip.  However, beyond that, the two. apps start to show their differences.

The Tesla app is nice because its built into the car, and you can start the navigation on your phone, then send it to the car before you get in.  The way Tesla has built their ecosystem it feels like this is how you are supposed to do it.  The biggest limitation is that its built for you to plot your course and go now, and it also seemed to demand the full potential of the car too.  It would frequently use 80-90% of the battery to get you a longer distance, and it the charge times were much longer at each stop.  I didn't find a way to change this in the app, but its also possible I missed something.  The last gripe I have is that it doesn't show the amenities at each stop from the route screen; if you write down the chargers, then navigate back in the app you can see a row of icons indicating things like bathrooms, food, or shopping near by, but the app doesn't indicate if these things are walking distance from the charger.

ABRP on the other hand has a limitation in that it doesn't work natively on the car; it is a web page, so you can load it in the car's browser, but you lose some features if you do that, like Tesla's built in ability to prep the battery for the supercharger before you arrive at the station.  However, you can export the route to Google Maps, Apple Maps, or a Calendar, which are all ways that you can import the suggested stops into the Tesla's built in navigation system (Tesla also has a feature will it will automatically navigate to your next calendar appointment when you get in the car.)

ABRP also allows you to export the stops to a spreadsheet (or view the table in the browser) that outlines all the data you could want.  The battery charge you will have on arrival and charge you need to make the next station, estimated charging time, estimated cost, distance to the next station, and drive time.  This is basically an itinerary to review the trip and choose the stops you want to make.  This is also highly adjustable.  You can set the range of your vehicle on a full charge, set min and max limits for how much to charge the battery (I I try to keep it between 20 - 80%.). There is also an adjustment to set if you want more stops with shorter charges, fewer stops with long charges, or optimize for the quickest trip.

Once you create your route, you can click on each charging station to see what types of amenities are near by.  It will tell you if there are bathrooms, its dog friendly, has a playground, or is trailer friendly, and it will clarify how many food options you have with names, phone numbers, web sites, and a distance from charger.  The map view will use OpenStreepMap data to show you that the restaurant is only 800 ft away, but its across the interstate, so its more like a 1/2 mile walk.

This is a planning tool to help you know if its better to stop in Ellensburg and Mosses Lake, or push your battery a little more and go straight to Quincey.  A quick conclusion on that scenario will tell you there are multiple restaurants very close to the chargers in Ellensburg and Mosses Lake, but the one in Quincy is the 1/2 mile walk I talked about in the previous paragraph.  I also learned that there are two chargers in Ellensburg, one at 150Kw and another at 250Kw.  Now, those are both Tesla chargers, so it would be visible in the Tesla app too, but the tesla app seems to leave a lot to be desired during the planning phase of the trip.

So, you may have noticed I've left out the elephant in the room; most people use Google, Garmin and Apple maps for navigation.  Well, as a Tesla owner, there really isn't a convenient way to use any of those maps (well, technically the map data in a Tesla comes from Google.)  The big beautiful screen in the car doesn't support Android Auto or Apply CarPlay; a huge negative for any car these days.  So, for those who buy an electric car from any other car maker, you will have that experience at your finger tips.  However, you can't actually tell Google/Apple/Garmin maps to route your trip to include stops for electric chargers, calculate the charge time for those charger, or calculate the cost of charging.  However, the ABRP Android app uses Google Maps for its backend, and the Apple version of the app uses Apple Maps data; so that is worth mentioning.

Speaking of the app experiences, I think its clear at this point I am biased in favor of ABRP.  The Tesla experience for trip planning is okay, and its roughly the same in the web browser, mobile app, and on the car.  Actually, I think you get a little more google map data in the car than the phone or browser, but I think that's by design.  A great thing about using the phone is that all map programs have a nice feature to send a location directly to your tesla;  I use this all the time.  I often look up current travel time (including traffic) before I get in the car, so having a nice share button to tell my car to navigate there when I get in the car is amazing!  This is a tiny thing that makes me happy multiple times a day.

So, back to the ABRP mobile app.  I started by visiting the website on my computer and plotting my course.  I took that course and investigated which chargers had the amenities I wanted near by to help keep the kids and dogs happy, and now I am going back to the app to create a route with waypoints at my desired chargers.  I can also add a stop at a hotel.  This was very easy to do, and I created a free account to save my plan.  On my phone.

So, this concludes the planning phase of this road trip.  However, before the real trip, I am going to test out my theories with a smaller trip.  I will be faced with the decision:  can I count on my family to stick to a plan using calendar events for each charger?  The advantage here is that the Tesla would (in theory) sync my calendar, and load the next charger every time we get in the car.  However, this could fall apart and become unwieldy if we don't stick to a schedule; for the record, my family NEVER sticks to a schedule.  

The alternate plan would be to send the tesla the location of the next charger from my plan every time we leave a charger.  This might work, but could fall apart if cell service to either my phone or the tesla is unavailable.  We will just need to try and test it I guess.  Until the next phase.

Later,

     SteveO

The great webcam shortage of 2020...

So, among the plethora of missed opportunities that arose for the wise investor (like investing in 3B, Clorox, Moderna, Zoom., etc...)  I found another stock that would have been great to buy in March: Logitech.  Everywhere you look, all their webcams are sold out.  It actually got me to thinking about my own webcams that I've bought over the years...

I have a few laying around the house... I actually noticed that most of my webcams are logitech cameras, and it just turns out that I still have a few of them laying around the house.

In the early days of this blog, I had and old Logitech Quickcam.  I say old, but I have the thing I went digging in the garage and I found it.  You can see here the photo that I dutifully took in my photobox (affiliate link at the bottom...)

I tried digging for any of the old webcam photos or timelapse videos I used to have, but I can't find anything.  Maybe I'll loop back around and fill in more details.

It was an okay camera at the time, but the photos are complete garbage by today's standards.  I had some trouble with it because I think the USB cable is going bad after all this time 20+ years, 12 of which have been in a box wrapped up in a ball.

The next camera I managed to dig out of storage is a Logitech QuickCam S5500 (I think?)  As it turns out, Logitech doesn't put labels on these things, so I am just going by what my computer thinks it's called.

This is a better camera.  It has a higher resolution and a built in privacy guard that slides down over the camera sensor.  Honestly, it's about as good as the built in webcam in most laptops.  However, it's still pretty lame compared to more moden webcam.

I think I got this camera 12 or 13 years ago.  I replaced the QuickCam with it, and I continued to make timelapse videos out my window and save still images on a schedule with the program motion.  I just verified that it is still available in Ubuntu's repository too... good old apt install motion, and you are on your way.

It's funny to think that back then, I would save a still photo into a folder every second.  Those photos were stitched into a 14-hour timelapse, and every 5 seconds, the image was copied into a folder synchronized to dropbox and shared publicly.  Then I had a simple javascript utility that would refresh the 5 most recent right here on this blog.  ahh, those were some interesting days of manually writing HTML and XML documents to generate my resume with XSLT.

Finally, these days, I have a Logitech C922 on my computer.  I actually had a C920 at one point, and I got the corporate/lync/skype certified C930e for my Mom when she threw a baby shower for Renmin and I.  This form factor was used with a number of webcams.  This one supports 1080P video, and the corporate one has the same sensor, but a wider angle lense.  I also had one that was nearly identical to this camera, but it only supported 720P.

It's a pretty good camera, and in addition to video quality, the audio quality of the stereo microphones is pretty good too.  Some people have even told me it's better than my Plantronics headset, but I tend to use my Bose QC35+ headset during video calls, and people tell me that sounds great!

So I didn't just want to show you the cameras, I also setup a little photo shoot to compare them...

15+ year old QuickCam

The QuickCam has low resolution, has trouble dealing with the brightness of the lightbox, and you need to manually focus by turning the ring around the lense.  I also had trouble because I think the cable is going bad, and most photos I took turned out blurry even though I had the camera on a tripod.

12 year old QuickCam S5500

The QuickCam S5500 is a little better.  It actually has auto focus, and it had no trouble dealing with the bright lights in the lightbox.  You can even see the angle is wide enough to get the back corner of the box.  The two QuickCam models are actually in about the same spot, so you can see how much wider the angle is on the S5500.

5 year old Logitech C922

Next, I get to a more modern webcam.  Now, this camera has such a wide angle compared to the other two, that I had to move the camera in closer, and I still couldn't get rid of the back corners.






I guess just for good measure, I'll add a photo from my phone...

7 month old Pixel 3 XL smartphone

It really is amazing to look at how far digital cameras have come in the past 20 years.  I remember in 2000 it has a huge deal when we had a digital camera that saved BMP images is a 1.4MB floppd disk.  It was so cool that we could just carry around this "small" 3.5" disk and save a few pictures on it.

Now I'm not happy if a camera doesn't support 4K video or HDR.  How times have changed.


Anyway, It's time for me to get some sleep, so Goodnight...

Later,

SteveO

Virtual Interview Difficulties, and my plan to make it better.

Hey, so I guess I have had some trouble getting back into the idea of blogging.  Maybe I can make a recovery and not just rant about politics, but I don't know...

I've had a lot of stress in my life recently with: my boss retaliating against me for the crime of having a child; then being forced to work for the county for 6 months under the threat of paying over $25,000 if a quit (a type of debt slavery); then finally being able to quit, but dealing with my offer being retracted everyone stopped hiring because of this virus.  If not for my totally awesome family and that COVID stimulus check, I would be totally screwed right now (instead I still have a fleeting hope of making through this in one piece while my wife works).  Anyway, enough about me, I made my bed, I can work it out.

It wasn't all bad though.  I went through a round of interviews for Google.  It was an Software Engineer - Engineering Productivity interview (SWE-EngProd).  I was super excited by the idea of what I could do at the job, but unfortunately, I didn't make it through the hiring committee.  This is the second time I have made it this far in a Google Interview, but this time I really had a hard time with the virtual online interviews.  My excuse is the lack of a white board.Which has got me thinking about my next interview.  So... I got an idea...  If I practice like this:

Then maybe I should interview like this:

I don't know, maybe it's crazy to think like that ;P.

Regardless, I have an Amazon interview coming up, and I have been working on how I could do camera switching with Chime, and I came up with this idea.

I had purchased that overhead camera a while ago on Amazon just to see if this might work.  At the time I was thinking about creating an online YouTube class, and all the MOOCs I saw on Coursera and Udacity featured a video of a hand drawing on a white board.  So, I thought, why can't I do that?  I explain crap all the time to people.  So I bought it.

Now, the camera mount I got isn't available anymore, but the identical item can still be purchased under a different name.  Here is an affiliate link if you think you want to buy one of these:(I.E. I get paid if you click the link and buy one.)  Anyway, here are links to the "Photo Copy Stand" and the webcam I have connected to it:

Full disclosure, I used a cheap little dry erase board I had laying around, but I did just buy a 48"x36" magnetic dry erase board for the back of my office door, and it works great with those magnetic dry erase markers.  I am normally an expo marker person, but the Amazon bookstore in the mall had those in the checkout isle, so I grabbed it while getting something else.

I also used those cool little 2040 Design Studio Pugoo pads, where are these awesome sticky pads that I use as a non-permanent adhesive, but it looks like that company doesn't exist anymore.  So, I can't shill for them too.  That's right people, I am selling out by putting an advertisement on my blog. What else will unemployment drive me to do?

I promise not to go too crazy with the advertising, but I think the context here works, so I'm going to do it.  Oh, about that $165 webcam, that price is inflated due to COVID.  Normally, that's a $70 webcam, but because everyone is working from home, laptop webcams suck, and most people don't like the new $200+ webcams, this older but still pretty good webcam is in much higher demand that normal.  So, unless you are desperate, wait for Logitech to pick up production again.

ok, so, I'm trying my hand at being a corporate shill, but at least I'm promoting some things I actually use... I think saying that will help me sleep at night.  Other than that, the family is doing good.  We still have the house, and I'm still interviewing for work, and that's about it for now.

Later,

     SteveO

Regarding Bernie Blackout

So, many of you may have heard about the recent Vice documentary Bernie Blackout.  I recently saw it, and I have to say it's nice to know other people noticed the same thing as me.  I think they sum it up best with this one chart I grabbed from the show:

Screen clipping from the Vice Bernie Blackout Documentary.

I think it's really telling that every single time one of the news networks started talking about Bernie, it was phrased in a negative light.  For example, all candidates had about 2% of their supporters talk negatively about others online, but because Bernie had more supporters than all others combined, this lead to the invention of a false "Bernie Bro" narrative.  If he won a state, then it was somehow the smallest win in history.  They call this the Carl Rove strategy in the documentary, and I guess so, I don't really know to be honest.

Then there are the three spikes that occur in that bar graph: one the day before super tuesday, one at the end of super tuesday, and one on April 9th.  These are interesting because the largest spike, before super tuesday, the media chose to talk about everyone except Bernie.  Then the other two spikes were the two times where Bernie was talked about more than the others candidates; these were the day the media reported Bernie lost on super tuesday, and the day Bernie announced he was dropping out, proclaiming that Biden "won".

However, Biden didn't really win.  He was propped up by every other candidate that dropped out as well as the media narrative.  It's the same thing that happened with Hillary in 2016.  Bernie was filling stadiums with people he would bring to the Democratic Party; not the anti-Trump crowd that would vote Democrat no matter what, but people who have lost hope in our political system and would have voted for someone who actually believes what they are saying.  Those who are disenfranchised by people like Hillary and Biden.  The people who the Democratic Party doesn't care about.  People who actually believe in democracy.

Overall, I liked the documentary, and I think you should hop on over to Vice and watch it.  I felt that sort of tightness in your chest when someone is preaching to the choir of a repressed group; you know, it just kept tugging on my heart strings.  It pointed out a lot of things I felt were obvious.  Like the fact that people supported Bernie because he was promoting what they believed, not some evil old man putting ideas in their head.  It's why Hillary lost in 2016, and it's why I think Biden lost on April 9th.  Not because Bernie didn't support Biden, but because Biden (Hillary 2.0) doesn't stand for anything.

So, this is where I lose faith in our system again.  The same thing happened in 2016.  I had hopes for a candidate that would actually believe the purpose of government is to serve the people which is represents, but then we ended up with a choice between a douche and a turd sandwich.

Anyway... I guess that's my rant for now.

Later,

     SteveO

Adventures in the Land of Oracle and the Peoplesoft Test Framework

Hey everybody,

    I've been writing test cases with the PeopleSoft Testing Framework for the past year and a half, and I've found it hard to find any real objective feedback regarding how this framework operates and whether it's a tool that various businesses want to use.  So, I thought I would scribble my thoughts on the matter, and talk about testing with this interesting tool.  Oh, and just to be clear, I have a strong opinion that I am sure you will understand by the end.  (Scroll to the last line if you want to know the spoiler.)

   First, what is this thing about which I speak?  Oracle bought PeopleSoft some years back, and a tester on that PeopleSoft team wrote a UI automation tool to test new features when they come out.  He called this thing the PeopleSoft Testing Framework (which I will call PTF).  It is included with PeopleSoft, and it is a database-driven layer on top of Selenium as well as a UI to write/record and playback tests.

   When the County hired me to come on board and write their tests, I had never touched PeopleSoft before, and all I knew about it was that big companies use it for HR like stuff.  I think they hired me because the PTF UI had just enough clues for me to look at it, and see the similarities with Selenium development.

    Because I know Oracle gets a little litigious at times, I'll just describe the UI rather than post a screenshot.  The UI feels like a 90s era muti-document interface, complete with awkward elements that won't fit quite right on the page, and the inevitable empty spaces that almost always take up a third of the screen.  You have a sidebar that displays a folder tree along the left side, and inside that you will find tests and test shells.  The test shells are containers for you to execute tests from in a specific order.  The main meat of the program comes from tests - which look a lot like test shells - and are displayed as a table of values.  Each row is a test step, and the steps have some general properties like ID, comments, and a checkbox to toggle a step active/inactive; however, the meat of the tests is contained in the Type, Action, Recognition, Parameters, and Value fields.

     Type is used to point your action to a type of element on the page, the browser itself, or back at the PTF tool to run a function.  Each Type has a list of supported Actions (I.E. the type "button" has a "click" action.  The Recognition is mostly used for HTML element selectors, but there are exceptions to this, like the "Page" type, which uses GoTo and a unique identifier for the page.  You can pass Parameters to a special type of test call a library tests, and the "Get Property" Action on many Types will use Parameters to determine the exact property you want as well as what Variable you will store it as.  Finally, every test has a set of Test Cases, which use the Value field to create variants of the test.

Here is an example of what a test case would look like:

Type	Action	Recognition	Parameters	Value
Browser	Start_Login
Page	Prompt	ManagePosition.GBL		add
Page	PromptOk
Text	Set_Value	name=POSITION_DATA_EFFDT$0		&effdt

     The above, opens a new browser and logs in with a test user specified in the global PTF configuration (more on that later.)  it loads a page, with a value of "add", which actually loads a URL within the PTF environment (again in the global configuration), then clicks over to a section to create a new user.  PromptOk is a special action to click that add user command.  Finally, it enters text into a textbox on the page with the desired name.  The Value here is actually a variable, stored globally in the database.

     So, I said globally there a couple times.  PTF uses a global configuration for each environment; this locks down the base URL for the test, specifies a browser, and securely stores a username/password for use in your tests.  Also, because EVERYTHING is stored in the database, so are the variables, in fact, you can view your PTF variables using the regular PeopleSoft web UI.  There are also certain permissions in PeopleSoft security to allow a user to create, modify, or run PTF related things.  I'm not getting into that here.

     Now, I described the UI a lot, but if you go down the road to learn about PTF, everyone promotes the test recorder feature.  It does it's job well, but only works in Internet Explorer, and not even Microsoft wants to support that anymore; however, it will record what you click on and where you go.  There are also a few configuration options for making it recognize when you are just clicking through menus, and it will deactivate those steps in favor of a single goto action (however, it saves both methods to the test.

     Once you record some tests it is time to playback, which is the part about PTF I hate the most.  When you playback tests they run extremely slow.  I don't know if it's something the County has done or everyone has the same problem, but the tests are only capable of filling out about one text box per minute.  I have seen it take 5 minutes just to log into the system.  In theory, your tests would be running over night when nobody is sitting around watching the paint dry, but I have had a lot of problems getting command line execution to work correctly, and nobody can touch the keyboard or mouse while the tests are running.  I have also had a whole host or problems getting it to run in a Windows VM as well, and you can wave goodbye to Linux support.

    We found that things run well as long as we had a dedicated computer sitting in a corner cube to run the tests on, and we didn't mind leaving the computer unlocked when we walk away while still signed in... this part always makes me cringe.  I dread the day someone causes havoc while logged in with my credentials because of this issue.  I have found that if I log in using remote desktop, things work okay most of the time, but then I need to babysit the tests because we will occasionally get errors because it won't type in a value in some random text box; I suspect this is some sort of network related snafu, but it makes execution hard.  If you run in Hyper-V, get ready for random crashes and network failure messages, and god help you if you are connected to a VPN.

     As you might have picked up on at this point, running the tests requires constant babysitting.  I was told and once upon a bunch of the tests passed on to me ran smoothly, and nothing ever went wrong, but in my experience, this has never been the case.  You will run a testshell, that shell will call five test cases that run great, then the sixth one fails.  You review XML the log in PTFs internal log viewer and find that it didn't type in a required field, and the test failed half way through creating a complex object.  Fixing the problem looks like this:

1. You take the shell you are running, and deactivate the tests that succeeded
2. click into the failed test, make sure you are viewing the correct test case
3. deactivate whatever steps succeeded
4. possibly add an extra step to open the partially created object and edit it 
5. deactivate all the steps in the shell after the object is created
6. re-run the test... the mysterious failure goes away!
7. remove or deactivate the step that you possibly added to the test
8. re-activate all the stuff that was deactivated in step 3
9. deactivate everything that has been run in the shell
10. reactivate anything that was not run in the shell
11. run the shell again and pray you don't get another failure like that
12. possibly repeat steps 1-11 several times, but for a different failure in another part of the shell.

As a result of running these automated tests a few times, I know in detail all of the steps created in each script.  I know that if one part of the test fails, I need to manually go into the database and correct the data by hand with SQL or a compare report will fail.  I have an intimate knowledge of this kind of thing just from troubleshooting my tests, and saying.. well, lets see what happens if I just push forward with the tests even though that step failed... or I just repeat this step and create a new position rather than correcting an incorrect position.

Getting support from Oracle is painful.  You always get sent to an over-sea support center where they ask the same 10 questions before escalating your issue, meanwhile, you only get one message per day unless you decide to work in India's time zone; often times, issues will magically resolve themselves after a week with no action done by you. 

There is also no community support.  Or very little.  Most forum topics are asking for success stories with PTF, and the Oracle PTF focus group hasn't proven to be a source of information.  The only documentation Oracle has on the the tool is a chapter in their documentation on PeopleTools, and a $2,000 course that pretty much just walks you through the UI, and the teacher will not understand in-depth technical questions about the tool.

However, in the back of my mind, I keep looking back on my previous decade of experience before coming to work here.  I think back to the days of coding tests in Specflow/C#/Selenium... or Mocha/Webdriver.io/Javascript... heck, the other day, just for laughs I implemented a few of my tests in Robot Framework with Python... and that is when I lost all hope for PTF.

I admit, the idea of hiring a developer to write automated tests can be intimidating, but if you have a budget for using PeopleSoft, you have a budget for it.  The thing about Robot Framework specifically that really won my admiration at this point is actually the report it generates.  Just google RobotFramework tutorials on YouTube, or you might be able to access Lynda.com for free via your public library's website (you don't even need to leave home!)  Why do I happen to like it?  I'm glad you appear to care!

     Robot Framework has a nice keyword driven syntax.  So, you can start by describing your test in english, and gradually convert each sentence into code.  For example, start by writing a user-story like suite name  "As a ___ I want to do ___ so that ____ happens."  Then elaborate with a series of "Given ___ when someones does ____ then ____ happens" tests.  Each line of that file is a keyword, so, you can create a hierarchy of .robot files that define each keyword as a series of steps.  There are tons of libraries to add database, API, and even Selenium/Appium functionality to your tests, and if you really need custom functionality, everything is open source, and there are even tutorials on how to create your own custom libraries written in python.

     The tests support running locally, pointing at your own Selenium grid, or pointing to a cloud provider like browserstack of Saucelabs. You can even run multiple browser simultaneously.  However, best of all, is that report file.  It gives you a great summary that breaks down each file run into a bar graph of how many tests in that file passed/failed.  When you hit failures, clicking on the failed test opens a detailed log page, and you can drill down to the exact step that failed (complete with screenshots.)

     As a proper test automation framework, RobotFramework gives you the ability to write a custom piece of code that automatically run before/after all the tests as well as before/after each test.  Plus, there is a Jenkins plugin that perfectly integrates the whole thing into your build reporting pipeline.  Using a different CI server?  Don't worry, you just need to specify a folder to save the report in, which will contain everything you need.  results are in XML by default, but you can change formats if that works better for you.  I'm sure people will make more plugins for the Atlassian and Microsoft crowd too.

The final amazing part is that there is tons of documentation and an active community.  During the process of writing a few tests as a POC to compare against PTF, I found a link to almost everything I needed on the main website, and nearly all the github documentation pages followed the same format.  Nearly every question I came up with had an answer on Stack Overflow already.  The couple I had to ask were answered pretty quickly.

So, my conclusion is that it's really hard to compare PTF with a Selenium framework.  The theory behard PTF is that your PeopleSoft experts will be able to record tests and play them back rather than running tests manually, and sometimes it works that way, but my experience has not gone that way at all.  The reality has been that I need to fall back on my Selenium experience and web development background constantly.  I frequently end up using browser development tools for inspecting elements and exploring page DOMs to better define my selectors.  I've spent so much time debugging the tests, I constantly wonder if it really is faster than running the tests manually. 

That said, let me circle back for a second to this topic of learning curve, because that seems to be the #1 thing going for PTF at this point.  For that, I point you to Selenium IDE.  A wonderful tool that gives you almost all of the functionality of PTF.  Just record you tests in a browser, and you can play back the tests.  However, when troubleshooting, in both situations, you will need a lot of technical knowledge to parse through the depths.  So, if you are writing Selenium tests in C#, PTF might be easier, but it's not nearly as powerful.  Javascript is probably just as steep of a learning curve.  However, I think Python with Robot Framework is the easiest way to go.

TLDR:

PTF has poor documentation, little-to-no community support, inadequate corporate support, its limited to only controlling a single browser ata time, while connecting to a single PeopleSoft environment at a time, on a dedicated computer, and has no Continuous Integration server support.  On the other hand, Selenium has tons of documentation, a massive community, supports multiple simultaneous browsers, can handle multiple URLs to any site simultaneously, can run in the background on your workstation, and was built to be run from a Continuous Integration server.  Additionally, Selenium is way faster, and only a slightly steeper learning curve in some cases.
   

Anyway... I just had to vent for a few minutes... PTF is driving me nuts right now.

Later,

     SteveO
Spoiler:  I have grown to hate PTF and will never recommend people stray away from Selenium.