SQLITE3 Raspberry Pi Problems and Solutions?

29 thoughts on “SQLITE3 Raspberry Pi Problems and Solutions?”

1. I don’t understand the link between NR and NodeJS either. Perhaps it’s a feature that the two are independent. When I upgraded to NR 1.3.1 I checked the version of NodeJS on a lark. Good thing I did because it was version 8 and about to be unsupported by NR. So I bumped it up to v12 (the newest released version for long term support). Technically v8 was still OK for NR, but it seems NR should tell you during the upgrade that you might want to also upgrade NodeJS.

2. I unloaded everything to do with phplliteadmin and sqlite, removed everything I did with Mr.Shark’s instructions above. Then I redid the instructions above and I have blastoff!
   Now I just need to add the fields to the database.

   Thanks for the assistance.

   1. Glad you got it sorted, Scott

3. I’m trying to get follow along with a Node-Red flow with GPS data that uses sqlite and was advised to use phpliteadmin as it is “easy to use and set up”. Well, I’ve spent the every evening this past week trying to the phpliteadmin to work and I have not clue. Anyone have pointers or can send me in the right direction? I’ve not been able to find much in a search. I get to the webpage and then it asked to great a database, but then I get the error that sqlitDatabase not installed.

   RPi 4, Node-Red 1.2.5 and installed the sqlite/phpliteadmin with Synaptic.

   1. It’s such a long time since I installed SQLITE from scratch – I use “the script” to install NR, MQTT, SQLITE and the various NR nodes on a fresh Raspberry Pi. The only issues I’ve had, have been upgrades ro SQLITE node – but it looks like the upgrade to the NR install/upgrade script has fixed that. So – before thinking about NR, you need SQLITE itself in there. Never used Synaptec so can’t comment on that. phpliteadmin will create databases but only if the actual SQLITE itself is installed on the Pi. Possibly look at “the script” for pointers?

   2. extracting from the script, let’s say for debian 10: you need apache+sqlite:
      sudo apt-get -y install apache2 libapache2-mod-php7.3 sqlite3 php-sqlite3 php-xml php-mbstring

      then install phpliteadmin:
      cd /var/www/html
      sudo mkdir phpliteadmin
      cd phpliteadmin
      sudo wget –no-check-certificate https://www.phpliteadmin.org/phpliteadmin-dev.zip
      sudo unzip phpliteadmin-dev.zip
      sudo mv phpliteadmin.php index.php
      sudo mv phpliteadmin.config.sample.php phpliteadmin.config.php
      sudo rm *.zip
      sudo mkdir themes
      cd themes
      sudo wget –no-check-certificate http://bitbucket.org/phpliteadmin/public/downloads/phpliteadmin_themes_2016-02-29.zip
      sudo unzip phpliteadmin_themes_2016-02-29.zip
      sudo rm *.zip

      then configure it, manually going to search for these 3 parameters, or via sed and following lines (IMPORTANT in the 2nd one, change $adminpass to your wanted pass)
      sudo sed -i -e ‘s#\$directory = \x27.\x27;#\$directory = \x27/home/pi/dbs/\x27;#g’ /var/www/html/phpliteadmin/phpliteadmin.config.php
      sudo sed -i -e “s#\$password = \x27admin\x27;#\$password = \x27$adminpass\x27;#g” /var/www/html/phpliteadmin/phpliteadmin.config.php
      sudo sed -i -e “s#\$subdirectories = false;#\$subdirectories = true;#g” /var/www/html/phpliteadmin/phpliteadmin.config.php

      1. Thanks Mr.Shark, ran everything and nothing new was installed, the last 3 items produced this error:

         sudo sed -i -e ‘s#\$directory = \x27.\x27;#\$directory = \x27/home/pi/dbs/\x27;#g’ /var/www/html/phpliteadmin/phpliteadmin.config.php
         sed: -e expression #1, char 1: unknown command: `�’

         I made the changes with nano, but when I run the web application, the old password gets me in and not the new one and it still doesn’t let me create any databases.

         “There was a problem setting up your database, /var/lib/phpliteadmin/. An attempt will be made to find out what’s going on so you can fix the problem more easily.

         Checking supported SQLite PHP extensions…

         PDO: installed
         PDO SQLite Driver: installed
         SQLite3: installed
         SQLiteDatabase: not installed”

         Has to be something simple, where is it starting from perhaps?

      2. This appears to be the config file that is being used.

         /usr/share/phpliteadmin/phpliteadmin.php

         1. My config file is here – and my php works. /var/www/html/phpliteadmin/

         2. yes, you can use an external config file, or put all in main file as i do, it’s the same… don’t copy paste those lines because blog formatted them wrong, just look at them in the sed part to see what to search and how to replace that, are just 3 lines…

4. Asyou know, Leo, I’ve covered SSD and the Pi4 here – https://tech.scargill.net/ssd-cloning-on-raspberry-pi-4/ – possibly a better place to discuss SSD and speeds therein.

5. People first move over to SSD and when budget is OK change to Pi4. I made a clone of my freshly generated Debian in 3,5 minutes SSD>SSD clone on USB 3.
   I was trying to generate another fresh copy in a Pi 3 as a backup to my homeserver, but that hanged at installing Node_RED(half an hour was enough to kill the Script by pulling the power off). A SSD with USB3 of 120GB costs 23 euro. The benchmark shows also almost 300MB/s big chunk writing speed.

6. I have just spent a considerable amount of time pulling sqlite out of my systems and moved on to MariaDB due to these ongoing issues with the node red module. It was brought to a head when I wanted to move some of the flows to a HASSOS (Home Assistant OS) machine and discovered you cannot run the sqlite node on that OS because you need the make tools to compile and they are not available on HASSOS. That and the fact that the Home Assistant native log can get very large and uses sqllite by default. So I bit the bullet and upgraded and MariaDB works well, the Pi4 has no problem running the database server on top of everything else.

   1. Hi Steve

      Well,historically I moved to SQLITE and put that in my “The Script” because conventional wisdom said it was lighter on SD writes than MYSQL (hence Maria) – but but of course the goalposts have changed with SSD (and coming up in here this week, a small eMMC module to replace SD). I really think the people inolved in bringing SQLITE to Raspberry Pi (much as we all appreciate this stuff is most likely voluntary) need to rethink the compilation or make precompiled modules more readily available for SQLITE – I’ve lost more than one night’s sleep over the years with constant issues installing or upgrading SQLITE in Node-Red – almost all other nodes just update, no problem.

      I have Maria installed of course but never gotten around to modifying my queries to work with Maria/MySQL.

      I think first I’ll take a look at that comment above by Rinie Kervel then if all else fails spend the time to eliminate SQLITE.

7. The above represents a TINY fraction of the warning messages you see during the rebuild.
   Why not fix the compiler flags as the code runs fine?

   https://sqlite.org/forum/forumpost/44578d49b6:

   gcc -g -Wpedantic -Werror -Wall -Wextra \
   -Wsign-compare -fPIC -std=c89 -Wno-long-long \
   -Wno-implicit-fallthrough \
   -c -o sqlite3.o sqlite3.c

   1. I WAS going to come back with a quick answer “that’s fiine if you know how to do it” – then realised you’ve actually provided the answer by the look of it – well done.. it defeats me why the author left in such a level of warnings when the end user running his or her script can do nothing about them.

8. I added a 500GB disk drive and put my sqlite3 db on it. So far no problems.

   1. Excellent feedback. I’ve yet to put a large HD on the Pi. These notes however have enabled me to install/upgrade Node-Red on Buster on the Pi3 and now P4 and it all works without having to start from scratch.

      1. i see no point in using a 500gb hd, you’ll never use it fully with a raspberry (unless you use it as a nas, which, if not an rpi4 model, is not worth because of the shared bus between usb and eth), and at that point you can just use mysql or others which will not need to get a PITA every time you update your nodes 🙂

         1. Or Maria, even, after all it is standard with RPI now.

         2. The only reason I used the 500GB drive is that I had it on hand after replacing the hard drive ib a laptop with an SSD drive.

9. I could easily see me reverting to Maria (MySQL) as the SQLITE issues continue with no sign of a script fix. In Node-Red when updating, I frequently have to “rebuild” Sqlite3 (several minutes) which is a pain – and the build script is full of un-useful warnings about statements falling through.

   1. what about MongoDB? It uses json as native data format, it’s nodejs based (and you already have it), has excellent nodered support (AFAIK)…

      install: https://yannickloriot.com/2016/04/install-mongodb-and-node-js-on-a-raspberry-pi/

      web gui: https://webapplog.com/mongoui/

      comparison with sqlite and mysql: https://db-engines.com/en/system/MongoDB%3BMySQL%3BSQLite

      if you want to go for mysql, here some optimizations for PI: https://central.owncloud.org/t/mysql-mariadb-optimal-settings-for-running-on-raspberrypi-3-model-b/17004

10. If you are going to use databases on a RPi then you should use the more industrial SD cards.

    The very best SD cards use SLC (Single Level Cell) technology but are hard to find and very expensive – but often come with 5 year guarantees.

    MLC (Multi-Level Cell) based SD cards are easy to get and are more expensive than TLC (Triple Level Cell) cards but have much better endurance.

    The standard SD cards are usually TLC and are what is called consumer grade. Even then, some brands seem to work more reliably than others.

    I have been using Transcend 32GB High Endurance SD cards for a while and have no issue with MySQL databases.

    If your project and/or data has real value then you should use MLC wherever possible.

    1. Do you have specific known good makes and models in mind? I’ve not heard Transcend recommended before – it is usually Sandisk and Samsung that get mentioned as winners.

       1. Samsung Evo and Evo+ 32GB cards all the way down!

          My Pi2 has now been running several years on one and my Pi3 running for over a year. Even occasional power outages and forced power terminations (e.g. pull the plug) haven’t caused any issues at all.

11. was joking 🙂
    i mean, it’s known mysql corrupt sdcard on the long run, due to wear leveling and continuous writing… but sqlite too, with it’s long compile times and failing in this, and redoing and updating and what else, starts to be no good for sd, too 🙂

12. starting to think if it consumes more an sdcard a normal db like mysql or sqlite with all its problems to install and recompile… 😀

    1. Not sure I followed that…

Comments are closed.