Enphase Data Access

13567

Comments

  • System2
    System2 Posts: 6,290 admin
    Re: Enphase Data Access

    For those interested, I managed to pull the passwd file off of the pill shaped envoy.
    root:$1$$pNDuPsavJsKMV7LpLH29y.:0:0:root:/root:/bin/bash
    guest:!:506:506:Linux User:/home/guest:/bin/sh
    nobody:!:200:200:non-user:/dev/null:/bin/true
    ntp:!:105:105::/home/ntp:/bin/false
    energytracking:$1$$2UsWUEUB3Ky/53aZvd7r20:507:507:Linux User,,,:/home/energytracking:/bin/sh
    


    And here is the serial output on boot-up.
    Note: ctrl-c in redboot has been disabled.
    +++... Read from 0x0fee0000-0x0ff00000 at 0xc07e0000: .
    ... Read from 0x0fed3000-0x0fed4000 at 0xc07ff000: .
    FEC: [ FULL_DUPLEX ] [ disconnected ] [ 100M bps ]:
    ... waiting for BOOTP information0.
    Ethernet mxc_fec: MAC address 00:1d:c0:XX:XX:XX
    Can't get BOOTP info for device!
    Ethernet FEC MAC address: 00:00:00:00:00:00
    Clock input: 26 MHz
    Enphase Part Number: 590-00013-r01-v01.00.04
    
    Booting from [NOR flash]
    
    
    RedBoot(tm) bootstrap and debug environment [ROMRAM]
    Non-certified release, version FSL 200834 - built 16:12:28, Jul 29 2009
    
    Platform: Envoy (Freescale i.MX27 based) PASS 2.1 [x32 SDR]
    Copyright (C) 2000, 2001, 2002, 2003, 2004 Red Hat, Inc.
    Copyright (C) 2003, 2004, 2005, 2006 eCosCentric Limited
    
    RAM: 0x00000000-0x0ff00000, [0x00029200-0x0fed1000] available
    FLASH: 0xc0000000 - 0xc0800000, 64 blocks of 0x00020000 bytes each.
    == Executing boot script in 1.000 seconds
    RedBoot> emuwd set
    RedBoot> fis load -b 0x100000 kernel
    ... Read from 0x0fee0000-0x0feff000 at 0xc07e0000: .
    ... Read from 0x00100000-0x00300000 at 0xc0040000: ................
    RedBoot> exec -c "console=ttymxc0 root=/dev/mtdblock4 rootfstype=yaffs2 init=/sb
    in/init fec_mac=00:1D:C0:XX:XX:XX"
    entry=0xa0008000, target=0xa0008000
    Using base address 0x00100000 and length 0x00200000
    Uncompressing Linux.............................................................
    .............................................. done, booting the kernel.
    Linux version 2.6.24.7-ee404 (dhowson@dhowson-lnx) (gcc version 4.1.2) #1 PREEMP
    T RT Fri Jun 5 16:34:17 PDT 2009
    CPU: ARM926EJ-S [41069264] revision 4 (ARMv5TEJ), cr=00053177
    Machine: Enphase Energy MX27 Based Envoy
    Memory policy: ECC disabled, Data cache writeback
    CPU0: D VIVT write-back cache
    CPU0: I cache: 16384 bytes, associativity 4, 32 byte lines, 128 sets
    CPU0: D cache: 16384 bytes, associativity 4, 32 byte lines, 128 sets
    Real-Time Preemption Support (C) 2004-2007 Ingo Molnar
    Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 65024
    Kernel command line: console=ttymxc0 root=/dev/mtdblock4 rootfstype=yaffs2 init=
    /sbin/init fec_mac=00:1D:C0:XX:XX:XX
    MXC IRQ initialized
    PID hash table entries: 1024 (order: 10, 4096 bytes)
    MXC GPT timer initialized, rate = 13300000
    Console: colour dummy device 80x30
    Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
    Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
    Memory: 256MB = 256MB total
    Memory: 256424KB available (3040K code, 267K data, 120K init)
    Mount-cache hash table entries: 512
    CPU: Testing write buffer coherency: ok
    net_namespace: 76 bytes
    NET: Registered protocol family 16
    MXC WDOG1 Enabled
    AIPI VA base: 0xfc000000
    CPU is i.MX27 Revision 2.1
    Clock input source is 26000000
    MXC GPIO hardware
    SCSI subsystem initialized
    CSPI: mxc_spi-0 probed
    CSPI: mxc_spi-1 probed
    usbcore: registered new interface driver usbfs
    usbcore: registered new interface driver hub
    usbcore: registered new device driver usb
    NET: Registered protocol family 2
    IP route cache hash table entries: 2048 (order: 1, 8192 bytes)
    TCP established hash table entries: 8192 (order: 4, 65536 bytes)
    TCP bind hash table entries: 8192 (order: 6, 262144 bytes)
    TCP: Hash tables configured (established 8192 bind 8192)
    TCP reno registered
    MX27: Power management module initialized
    usb: DR host (usb3317) registered
    NetWinder Floating Point Emulator V0.97 (extended precision)
    krcupreemptd setsched 0
      prio = 98
    JFFS2 version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
    io scheduler noop registered
    io scheduler anticipatory registered
    io scheduler deadline registered
    io scheduler cfq registered (default)
    Serial: MXC Internal UART driver
    mxcintuart.0: ttymxc0 at MMIO 0x1000a000 (irq = 20) is a Freescale MXC
    console [ttymxc0] enabled
    mxcintuart.1: ttymxc1 at MMIO 0x1000b000 (irq = 19) is a Freescale MXC
    mxcintuart.2: ttymxc2 at MMIO 0x1000c000 (irq = 18) is a Freescale MXC
    mxcintuart.4: ttymxc4 at MMIO 0x1001b000 (irq = 49) is a Freescale MXC
    mxcintuart.5: ttymxc5 at MMIO 0x1001c000 (irq = 48) is a Freescale MXC
    RAMDISK driver initialized: 2 RAM disks of 40960K size 1024 blocksize
    loop: module loaded
    FEC ENET Version 0.2
    fec: PHY @ 0x1f, ID 0x0007c0c4 -- LAN8700
    eth0: ethernet 00:1d:c0:XX:XX:XX
    tun: Universal TUN/TAP device driver, 1.6
    tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
    Driver 'sd' needs updating - please use bus_type methods
    MXC MTD nor Driver 2.0
    Number of erase regions: 2
    Primary Vendor Command Set: 0001 (Intel/Sharp Extended)
    Primary Algorithm Table at 010A
    Alternative Vendor Command Set: 0000 (None)
    No Alternate Algorithm Table
    Vcc Minimum:  1.7 V
    Vcc Maximum:  2.0 V
    Vpp Minimum:  8.5 V
    Vpp Maximum:  9.5 V
    Typical byte/word write timeout: 256 µs
    Maximum byte/word write timeout: 512 µs
    Typical full buffer write timeout: 512 µs
    Maximum full buffer write timeout: 1024 µs
    Typical block erase timeout: 1024 ms
    Maximum block erase timeout: 4096 ms
    Chip erase not supported
    Device size: 0x800000 bytes (8 MiB)
    Flash Device Interface description: 0x0001
      - x16-only asynchronous interface
    Max. bytes in buffer write: 0x40
    Number of Erase Block Regions: 2
      Erase Region #0: BlockSize 0x8000 bytes, 4 blocks
      Erase Region #1: BlockSize 0x20000 bytes, 63 blocks
    mxc_nor_flash.0: Found 1 x16 devices at 0x0 in 16-bit bank
     Intel/Sharp Extended Query Table at 0x010A
     Intel/Sharp Extended Query Table at 0x010A
     Intel/Sharp Extended Query Table at 0x010A
     Intel/Sharp Extended Query Table at 0x010A
     Intel/Sharp Extended Query Table at 0x010A
    Using buffer write method
    cfi_cmdset_0001: Erase suspend on write enabled
    Searching for RedBoot partition table in mxc_nor_flash.0 at offset 0x7e0000
    4 RedBoot partitions found on MTD device mxc_nor_flash.0
    Creating 4 MTD partitions on "mxc_nor_flash.0":
    0x00000000-0x00040000 : "RedBoot"
    0x00040000-0x00240000 : "kernel"
    0x007e0000-0x007ff000 : "FIS directory"
    mtd: partition "FIS directory" doesn't end on an erase block -- force read-only
    0x007ff000-0x00800000 : "RedBoot config"
    mtd: partition "RedBoot config" doesn't start on an erase block boundary -- forc
    e read-only
    MXC MTD nand Driver 2.1
    NAND device: Manufacturer ID: 0x2c, Chip ID: 0xac (Micron NAND 512MiB 1,8V 8-bit
    )
    nand_read_bbt: bad blocks: 4, reserved blocks: 0
    Searching for RedBoot partition table in NAND 512MiB 1,8V 8-bit at offset 0x1ffe
    0000
    No RedBoot partition table detected in NAND 512MiB 1,8V 8-bit
    Creating 1 MTD partitions on "NAND 512MiB 1,8V 8-bit":
    0x00000000-0x20000000 : "nand.root"
    fsl-ehci fsl-ehci.0: Freescale On-Chip EHCI Host Controller
    fsl-ehci fsl-ehci.0: new USB bus registered, assigned bus number 1
    fsl-ehci fsl-ehci.0: irq 56, io mem 0x10024000
    fsl-ehci fsl-ehci.0: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
    usb usb1: configuration #1 chosen from 1 choice
    hub 1-0:1.0: USB hub found
    hub 1-0:1.0: 1 port detected
    Initializing USB Mass Storage driver...
    usbcore: registered new interface driver usb-storage
    USB Mass Storage support registered.
    usbcore: registered new interface driver libusual
    mice: PS/2 mouse device common for all mice
    pcf2123 spi driver
    pcf2123 spi1.0: rtc core: registered pcf2123 as rtc0
    MXC WatchDog Driver 2.0
    MXC Watchdog # 0 Timer: initial timeout 60 sec
    MXC MMC/SD driver
    mxcmci-0 found
    oprofile: using timer interrupt.
    TCP cubic registered
    NET: Registered protocol family 1
    NET: Registered protocol family 17
    RPC: Registered udp transport module.
    RPC: Registered tcp transport module.
    ieee80211: 802.11 data/management/control stack, git-1.1.13
    ieee80211: Copyright (C) 2004-2005 Intel Corporation <jketreno@linux.intel.com>
    Envoy Post Initialization
    pcf2123 spi1.0: setting system clock to 2010-07-09 23:08:57 UTC (1278716937)
    yaffs: dev is 32505860 name is "mtdblock4"
    yaffs: passed flags ""
    mmc0: new SD card at address 8fe4
    mmcblk0: mmc0:8fe4 SU02G 1931264KiB
     mmcblk0: p1
    VFS: Mounted root (yaffs2 filesystem).
    Freeing init memory: 120K
    UART 1 has been activated multi-times
    INIT: version 2.85-ts1.01 booting
    kernel.panic = 2
    vm.panic_on_oom = 1
    Starting the hotplug events dispatcher: udevd.
    Synthesizing the initial hotplug events...done.
    Waiting for /dev to be fully populated...done.
    INIT: Entering runlevel: 3
    Setting hostname....
    Bringing up interface eth0..
    Determining IP information for eth0....udhcpc (v0.9.9-pre) started
    udhcpc[1645]: udhcpc (v0.9.9-pre) started
    adapter index 2
    udhcpc[1645]: adapter index 2
    adapter hardware address 00:1d:c0:XX:XX:XX
    udhcpc[1645]: adapter hardware address 00:1d:c0:XX:XX:XX
    vforking and execle'ing /usr/share/udhcpc/default.script
    udhcpc[1645]: vforking and execle'ing /usr/share/udhcpc/default.script
    eth0: config: auto-negotiation on, 100FDX, 100HDX, 10FDX, 10HDX.
    entering raw listen mode
    udhcpc[1645]: entering raw listen mode
    Opening raw socket on ifindex 2
    udhcpc[1645]: Opening raw socket on ifindex 2
    adding option 0x35
    udhcpc[1645]: adding option 0x35
    adding option 0x3d
    udhcpc[1645]: adding option 0x3d
    adding option 0x0c
    udhcpc[1645]: adding option 0x0c
    adding option 0x3c
    udhcpc[1645]: adding option 0x3c
    Sending discover...
    udhcpc[1645]: Sending discover...
    Waiting on select...
    udhcpc[1645]: Waiting on select...
    adding option 0x35
    udhcpc[1645]: adding option 0x35
    adding option 0x3d
    udhcpc[1645]: adding option 0x3d
    adding option 0x0c
    udhcpc[1645]: adding option 0x0c
    adding option 0x3c
    udhcpc[1645]: adding option 0x3c
    Sending discover...
    udhcpc[1645]: Sending discover...
    Waiting on select...
    udhcpc[1645]: Waiting on select...
    eth0: status: link down.
    adding option 0x35
    udhcpc[1645]: adding option 0x35
    adding option 0x3d
    udhcpc[1645]: adding option 0x3d
    adding option 0x0c
    udhcpc[1645]: adding option 0x0c
    adding option 0x3c
    udhcpc[1645]: adding option 0x3c
    Sending discover...
    udhcpc[1645]: Sending discover...
    Waiting on select...
    udhcpc[1645]: Waiting on select...
    vforking and execle'ing /usr/share/udhcpc/default.script
    udhcpc[1645]: vforking and execle'ing /usr/share/udhcpc/default.script
    No lease, forking to background.
    udhcpc[1645]: No lease, forking to background.
    udhcpc[1649]: Waiting on select...
     done....
    Bringing up interface lo..
    Starting system log daemon: syslogd.
    Starting NTP server: ntpdStarting OpenBSD Secure Shell server: sshd.
    Starting periodic command scheduler: cron.
    Starting Envoy System Monitor...done
    EMU database exists:
    -rw-r--r--    1 root     root       142336 Jul  9 15:51 /var/opt/emu/emu.db
    Loading emuk2u...Using /lib/modules/2.6.24.7-ee404/kernel/drivers/emuk2u/emuk2u.
    ko
    done
    LCD Timeout
    Starting emu...done
    
    
    Technologic Systems TS-LINUX/arm 7.0
    envoy login:
    
  • sandeen
    sandeen Solar Expert Posts: 48 ✭✭✭
    Re: Enphase Data Access
    nbias wrote: »
    For those interested, I managed to pull the passwd file off of the pill shaped envoy.

    Nice job! If it changes in the future we can assume Enphase is watching the thread. :)
  • dwh
    dwh Solar Expert Posts: 1,341 ✭✭✭
    Re: Enphase Data Access

    Those two hashed passwords are crackable.
  • sandeen
    sandeen Solar Expert Posts: 48 ✭✭✭
    Re: Enphase Data Access

    Not that it has much to do with actual data access, but some might be interested in this.

    Enphase has posted some of the source code to a couple of GPL'd components that they use at http://www.enphaseenergy.com/support/patches.cfm

    Seems to be a relatively new page; google can't find any inbound links anywhere ;)

    When Enphase first pointed me at this page, it contained only the kernel tarballs, and neither link worked. They work now, but so far I can't uncompress the first kernel .tbz file, which should be a bzip2'd tarball.

    The redboot image just showed up today, after I sent them a rather long list of the GPL'd software in their device (see below), in response to an assertion that they had nothing modified but the kernel. (There seemed to be a notion that distributing binaries built from unmodified source released them from large swaths of the license).

    The kernel tarball is timestamped from February, and the redboot image has today's date.

    To their credit, the redboot source does actually seem to be their modified source; the config directive which disables ctrl-c seems to be a local change:
    # ./fconfig -l -d redboot_config
    Low verbosity messages are printed.
    boot_no_break: TRUE
    ...
    

    and it is present in the tarball:
    RedBoot_config_option("No break in boot script",
                          boot_no_break,
                          ALWAYS_ENABLED, true,
                          CONFIG_BOOL,
                          false
        );
    

    If they had posted a pristine upstream tarball that would have been quite annoying. :) I'll have to chase through this a little more to see if there is some alternate to ctrl-c when it's config'd this way.

    They're making an effort here, which is good, I guess, but there is a ways to go yet ;)

    Here's a (possibly incomplete and/or inaccurate) list of what appears to be GPL'd software in the unit, and the licenses of those packages:
    Package: bash; 		File: /bin/bash 		License: GPLv3+
    Package: busybox; 	File: /bin/busybox 		License: GPLv2
    Package: coreutils; 	File: /bin/chroot 		License: GPLv3+
    Package: coreutils; 	File: /bin/touch 		License: GPLv3+
    Package: diffutils; 	File: /usr/bin/diff 		License: GPLv2+
    Package: e2fsprogs; 	File: /sbin/e2fsck 		License: GPLv2
    Package: e2fsprogs; 	File: /sbin/mke2fs 		License: GPLv2
    Package: findutils; 	File: /usr/bin/find 		License: GPLv3+
    Package: glibc-common; 	File: /usr/bin/ldd 		License: LGPLv2+ and LGPLv2+ with exceptions and GPLv2+
    Package: logrotate; 	File: /usr/sbin/logrotate 	License: GPL+
    Package: lrzsz; 	File: /bin/lrz	 		License: GPLv2+
    Package: lrzsz; 	File: /bin/lsz	 		License: GPLv2+
    Package: module-init-tools; 	File: /sbin/depmod 	License: GPLv2+
    Package: module-init-tools; 	File: /sbin/modinfo 	License: GPLv2+
    Package: mtd-utils; 	File: /usr/sbin/flashcp 	License: GPLv2+
    Package: mtd-utils; 	File: /usr/sbin/flash_lock 	License: GPLv2+
    Package: mtd-utils; 	File: /usr/sbin/flash_unlock 	License: GPLv2+
    Package: nano; 		File: /usr/bin/pico 		License: GPLv3+
    Package: net-tools; 	File: /bin/netstat 		License: GPL+
    Package: openvpn; 	File: /usr/sbin/openvpn 	License: GPLv2
    Package: procps; 	File: /bin/ps 			License: GPLv2+ and LGPLv2+
    Package: procps; 	File: /sbin/sysctl 		License: GPLv2+ and LGPLv2+
    Package: procps; 	File: /usr/bin/top 		License: GPLv2+ and LGPLv2+
    Package: rsync; 	File: /usr/bin/rsync 		License: GPLv3+
    Package: ruby; 		File: /usr/bin/ruby1.8 		License: Ruby or GPLv2
    Package: setserial; 	File: /bin/setserial 		License: GPL+
    Package: SysVinit; 	File: /bin/mountpoint 		License: GPLv2+
    Package: SysVinit; 	File: /sbin/init 		License: GPLv2+
    Package: SysVinit; 	File: /sbin/reboot 		License: GPLv2+
    Package: SysVinit; 	File: /sbin/shutdown 		License: GPLv2+
    Package: udev; 		File: /sbin/udevcontrol 	License: GPLv2
    Package: udev; 		File: /sbin/udevd 		License: GPLv2
    Package: udev; 		File: /sbin/udevsettle 		License: GPLv2
    Package: udev; 		File: /sbin/udevtrigger 	License: GPLv2
    Package: udev; 		File: /usr/bin/udevinfo 	License: GPLv2
    Package: udev; 		File: /usr/bin/udevtest 	License: GPLv2
    Package: udev; 		File: /usr/sbin/udevmonitor 	License: GPLv2
    
    Libraries:
    
    Package: pam;		License: BSD and GPLv2+
    Package: glibc;		License: LGPLv2+ and LGPLv2+ with exceptions and GPLv2+
    Package: procps;	License: GPLv2+ and LGPLv2+
    Package: libcap;	License: LGPLv2+ or BSD
    Package: libattr;	License: LGPLv2+
    Package: gpm-libs;	License: GPLv2+
    Package: readline;	License: GPLv3+
    Package: libsepol;	License: LGPLv2+
    Package: libvolume_id;	License: GPLv2
    
  • andyman
    andyman Solar Expert Posts: 32
    Re: Enphase Data Access

    Wow glad someone posted this info. Thanks everyone for the hardwork. Should we post a guide to actually break into the thing via JTAG so everyone can play with it? Well whoever wants to get in without waiting on the crack for the password, order yourself this...

    http://www.sparkfun.com/commerce/product_info.php?products_id=8278

    You'll also need to download the latest version of openocd, easiest if you used linux.



    FYI that password is a salted FreeBSD MD5 hash. It'll take a while to crack.

    Also FYI... But you already know this. They are tunneling out via openvpn from inside your network to theirs. Its a bi directional tunnel so theoretically they have access to your secured internal network. First time I've ever seen a device do that, not very cool. They coulda just done an http put or something.

    andy
  • dwh
    dwh Solar Expert Posts: 1,341 ✭✭✭
    Re: Enphase Data Access
    andyman wrote: »
    Should we post a guide to actually break into the thing via JTAG so everyone can play with it?

    Probably shouldn't post it here. Discussing possibilities and sharing information is one thing, posting a HOWTO is something else. Let's not borrow trouble for NAWS.

    FYI that password is a salted FreeBSD MD5 hash. It'll take a while to crack.

    It's all relative. Back in the early '90s my cluster of 386s took a WHILE to run a dictionary attack against hashed passwords..."a while" meaning anywhere from 1 to 6 months.
  • drees
    drees Solar Expert Posts: 482 ✭✭✭
    Re: Enphase Data Access
    dwh wrote: »
    Probably shouldn't post it here. Discussing possibilities and sharing information is one thing, posting a HOWTO is something else. Let's not borrow trouble for NAWS.
    How would it be borrowing trouble? Reverse engineering is not illegal.
  • dwh
    dwh Solar Expert Posts: 1,341 ✭✭✭
    Re: Enphase Data Access
    drees wrote: »
    How would it be borrowing trouble? Reverse engineering is not illegal.

    Not that it *would* be borrowing trouble, but that it *could* be borrowing trouble.

    Do we really want to put NAWS in a position of having to explain to Enphase that "reverse engineering is not illegal"? Be a bit rude of us to put NAWS in that position.

    Better if the issue never comes up, no?
  • andyman
    andyman Solar Expert Posts: 32
    Re: Enphase Data Access

    Ok I won't post any guides, don't wanna get anyone in trouble. If anyone really needs to get in they can PM me.
  • CORNWALLAV8R
    CORNWALLAV8R Registered Users Posts: 5
    Re: Enphase Data Access

    Ok guys you have me supremely stoked. I bought 44 Enphase Model 190's, have 44 230 watt panels on order, will be here next Thursday, and I have no interest in paying a king's ransom for my own data. Ridiculous.

    Ideally we should be able to monitor and display the Envoy recorded data on our own PC, WITH AN OPTION to allow its use on Enphase's website for others to see. Hijacking our own individual data is too much.

    Am placing an order for (2) 5kW 2-axis trackers, they unfortunately won;t be here till after Christmas. Something to work on in the dead of winter I guess.

    Keep at it guys (and gals?) I wish I had more software experience, mostly just some embedded assembler and C.

    Cheers
  • Peter_V
    Peter_V Solar Expert Posts: 226 ✭✭✭
    Re: Enphase Data Access
    andyman wrote: »
    Ok I won't post any guides, don't wanna get anyone in trouble. If anyone really needs to get in they can PM me.

    Have you actually been able to access the data? I'd love to be able to do that and trackit on my own server instead of having to go through enphase.

    I'd appreciate any help you, or anyone, can offer on accessing the data directly on the emu or even from the inverters.
  • andyman
    andyman Solar Expert Posts: 32
    Re: Enphase Data Access

    Hey guys,
    Data access isn't so easy if you don't know how to use a JTAG interface. The instructions can get a little involved and I can't teach every single one of you guys how to do it. And if you fry your unit I don't want to be responsible.

    But for those that are already in, data access is easy. Obviously this is the raw data and unless you have a server, its really not worth it. Envoy is a nice service and gives you data that looks pretty.

    However if you really must host your own data like I do.. Then heres how ya do it. You can do with the data as you please.

    The database you care about is in /var/opt/emu/binned/binned.db and
    /var/opt/emu/emu.db

    To get a list of device serial numbers
    cd /var/opt/emu
    sqlite3 emu.db "select * from eh_device"

    more importantly to get a list of device_ids which are mapped to the device serials
    sqlite3 emu.db "select * from eh_channel"


    Once you have a device_id you can start pulling data out.
    To get the entire history
    sqlite3 binned.db "select * from channel_perf_interval where interval_type == 1 AND device_id == <type your device id here.. ie 823432322 " > out.txt

    With the file out.txt you can use excel or gnu plot to graph your data.

    My bash script for quickly getting a graph
    #!/bin/bash
    xlabels=minutes
    #title=$1
    ylabels="power (watt)"
    #inputfile=$1
    title="Power Used in watts"
    wattscript()
    {
    echo -E "set title \"$title\""
    echo -E "set xlabel \"$xlabels\""
    echo -E "set ylabel \"$ylabels\""
    echo "set terminal png"
    echo -E "set output \"$file.png\" "
    echo -E "plot \"$file.csv2\" with lines"
    }

    for file in $1
    do
    title=$file

    # $9 'stat1' => 'energy produced',
    # $10 'stat2' => 'energy consumed',
    # $11 'stat3' => 'rms voltage',
    # $12 'stat4' => 'rms current',
    # $13 'stat5' => 'peak voltage',
    # $14 'stat6' => 'peak current',
    # $15 'stat7' => 'power factor',
    # $16 'stat8' => 'reactive energy',
    # $17 'stat9' => 'stat9'

    tail -n 488 $1 | awk -F "|" '{ print $4,$9}' | sed 's/|/ /g' > tmp.csv
    sed '1d' tmp.csv > $file.csv2
    wattscript > tmp.plt
    gnuplot tmp.plt
    rm $file.csv2
    rm tmp.csv
    rm tmp.plt
    done






    If you want to pull out just a single piece of data for each device you can run
    sqlite3 binned.db "select stat1 from channel_perf_interval where oid = (select max(oid) from channel_perf_interval where device_id == 8834xxx ) "


    That will output the last energy produced and you can simply input that to any program like cacti for graphing on a 5 min poll interval.
  • andyman
    andyman Solar Expert Posts: 32
    Re: Enphase Data Access

    Now I personally won't to pay for my own data and I certainly don't ever want this device on my network because its a backdoor VPN directly into your home network (I secure my network) but Enphase does offer more then just your data with a subscription. I believe they offer a gaurantee of your unit and if it ever fails they will pay you for the electricity you would have generated. That is pretty cool. Not to mention the interface is pretty.

    I think most people will pay anyway so I don't see why they feel the need to lock people out.




    andy
  • Ralph Day
    Ralph Day Solar Expert Posts: 1,019 ✭✭✭✭
    Re: Enphase Data Access

    Andy,
    I never leave my system un-attended, if I'm not using it either it's in hibernate or standby. Are you saying that someone could hack/piggyback through the Envoy and get access to my laptop?

    Any particular security features to consider in firewalling the Enphase link? I suppose the ultimate would be to access the Envoy only by plugging in the ethernet cable once per day.

    Ralph
  • andyman
    andyman Solar Expert Posts: 32
    Re: Enphase Data Access

    The envoy unit uses openvpn to make a secure tunnel originating from your network to theirs. They can then potentiallly initiate new connection from enphases side to get into the envoy. They do this to poll the data and do software updates.

    But what this also does is basically give someone access to your local lan. Its as if I am in your house on your network. U can hibernate or put your system on standby but what if you were home and working. I could do a range of stuff once I'm in. You can unplug the enphase but that defeats the purpose of data gathering.

    The real solution is to put the enphase on the DMZ but not many people have a sophisticated network setup with a segregated lan (most people don't even know how to setup a routed subnet).

    Now I'm not saying anyone can hack this device. What I'm saying is someone at Enphase have accesss to your home network. I doubt they will do much but the potential is there.
  • CORNWALLAV8R
    CORNWALLAV8R Registered Users Posts: 5
    Re: Enphase Data Access

    If it is just Enphase with access I have no worries at all. A super company which clearly wants a good reputation. I wouldn't worry about it. What are they going to do, break into my PC and steal my wife's cookie recipe? Like they even have time to snoop around on our lans.
  • BB.
    BB. Super Moderators, Administrators Posts: 33,431 admin
    Re: Enphase Data Access

    If you want tin foil hat time--It is not unknown for the government to get a search warrant and have OnStar listen to conversation in the car.

    Also, there is a law that digital phone switches be designed to to facilitate wire taps. The first Rolm digital switches were not designed to support wire taps and were very popular with certain banks and "businesses" because of that because nobody (at the time) could tap the digital phone lines without "breaking" the digital phone connection.

    Later switches were designed to very do taps very nicely when given the correct commands (from what I recall from people who worked there decades ago).

    It is also not unknown for government ordered tap hardware/software to be hijacked by others for spying/etc.

    Could Enphase get a warrant that required some sort of network hacking software to be downloaded or could somebody hack a unit to do this--probably is possible.

    Do people spend huge amounts and money to hack computer networks and PCs? Yep. Is there a large enough install base for somebody to actively hack an Enphase unit--certainly not at the present time.

    Would somebody spend the time and money to do this for your wife's cookie recipe? Does she have great cookie recipes? ;)

    -Bill
    Near San Francisco California: 3.5kWatt Grid Tied Solar power system+small backup genset
  • CORNWALLAV8R
    CORNWALLAV8R Registered Users Posts: 5
    Re: Enphase Data Access

    Interesting thoughts. And yes, she has great cookie recipes, but it is all in the making, not so much in the recipe. Baking is not easy :-)

    I guess we can look at this in at least 2 ways. One, I have nothing to hide, so don't give a darn. But second, it makes sense to limit exposure to unnecessary risk. A bit of DMZ implementation is a good idea I guess. Wish I had more expertise in this stuff.
  • drees
    drees Solar Expert Posts: 482 ✭✭✭
    Re: Enphase Data Access

    A DMZ is a good idea - the problem is that your typical home router equipment does not typically have more than 2 subnets - so setting one up is really only done by people with a good amount of computer networking experience.
  • andyman
    andyman Solar Expert Posts: 32
    Re: Enphase Data Access

    Lets not distract the conversation of data gathering of our panels via enphase with network security. Because soon we'll get into bill of rights, 2nd and 4th amendment issues then we'll really go off on a tangent.
  • CORNWALLAV8R
    CORNWALLAV8R Registered Users Posts: 5
    Re: Enphase Data Access

    So andyman, if I understand correctly, we either need to wait for a password fix, or buy and use the JTAG programmer and download OpenOCD to get into the device to access the logs.

    I am patient and don't need the access right away, so if there is progress on the former, please let us all know. And a more detailed how to, would be most appreciated, even if it is supplied offline. Some of us otherwise very technical people are capable of following instructions but don't have the particular experience you do. :-)
  • andyman
    andyman Solar Expert Posts: 32
    Re: Enphase Data Access

    Ok I guess I'm still a little puzzled by the stats. I dug a little deeper and it looks like the way the EMU works is that there is a daemon called emu /opt/emu/bin/emu runs constantly. This software is what either polls data or gathers data from the actual inverters and its output is an sqlite database as I've been referring to located on /var/opt/emu

    Now the actual data for each panel is stored in binned/binned.db under the channel_perf_interval tables.

    What I don't understand is what is stat1 (11793). Its definitely not watts.

    ******** update *****

    Since I don't know what stat1 is I've decided its easier just to multiply the DC voltage with the current (stat4 * stat5 /1000000) = watts. I then set my graph up such that Y has a max of 200watts. Using mrtg to graph. I've now got a virtual display of how all my panels are doing and how shade affects each panel. Not as pretty as enphases flash but it will get the job done perfectly.


    andy

    ********************


    A sample line of that table is below.
    783161|1|822329601|1290788400|300|300|1|0|11793|244510|60050|30833|1375|12|0|0|1290788206


    where
    item 1 = 783161 = interval_id
    2 interval type
    3 device_id
    4 interval_duration (300 )
    5 stat_duration (happens to be 300)
    6 complete
    7 exported
    8 stat1 (11793) energy produced
    9 stat2 (244510) 244 volts AC voltage
    10 stat3 (60050) 60.050 hz
    11 stat4 (30833) 30.833V DC voltage
    12 stat5 (1375) 1.375A DC current
    13 stat6 (12) 12 Celcius
    14 stat7 (0) errors_sec
    15 stat8 (0) max error cycles
    16 created_date 1290788206
  • System2
    System2 Posts: 6,290 admin
    Re: Enphase Data Access

    Hacking the Envoy EMU is pretty brilliant... however, if it represents a risk to your home network, what about directly polling the micro-inverters? This would open up a whole new vista of possibilities for mining your own data without the need of the EMU.
  • andyman
    andyman Solar Expert Posts: 32
    Re: Enphase Data Access

    You can't access the inverter units directly. We don't have the binaries to do it. Once you hack it, its no big deal. Just rename the tunnelout scripts and it can't tunnel out and your basically secure. You can also change the ssl keys so it can't authenticate to enphase. Its easy to secure the device from outbound connections.
  • System2
    System2 Posts: 6,290 admin
    Re: Enphase Data Access

    Are you so sure it "can't be done"? The inverters have to be fairly sophisticated themselves... some arrays have hundreds of them. In that case, they probably have to do some sort of CDMA, probably have to have a MAC address, or an equivalent, and have a common command and control language. Sophisticated, yet simple, because ENPHASE can tunnel in and change the voltage and frequency tolerances if necessary (Enphase told me this). The EMU is the big security blockade. The inverters, because there's no telling what EMU they will be attached to, are likely unencrypted and simple to communicate with. Break the communications protocol, watch the data flow, and in the end, throw away the EMU.
  • Peter_V
    Peter_V Solar Expert Posts: 226 ✭✭✭
    Re: Enphase Data Access
    andyman wrote: »
    You can't access the inverter units directly. We don't have the binaries to do it. Once you hack it, its no big deal. Just rename the tunnelout scripts and it can't tunnel out and your basically secure. You can also change the ssl keys so it can't authenticate to enphase. Its easy to secure the device from outbound connections.

    As I understand it the EMU is already setup to send data about every 5 minutes to enphase.

    If you can access the EMU, how hard would it be to setup a DNS to redirect it's output to your own secure server? You could replace the ssl keys with your own self generated ones. All you'd have to figure out is the data format for the reports.

    Seems like, once you figure it out, this would be easier than logging into the emu and extracting the data.
  • andyman
    andyman Solar Expert Posts: 32
    Re: Enphase Data Access

    They use openvpn to create a tunnel back to corporate. Even if you intercepted with dns redirection you don't have the private keys for the tunnel so pointless. All you'll do is secure the device which you can already do via firewalling the inside address.

    Anyway I've attached one of my graphs from my panel. Its in watts. I have an html page with all my panels layed out in the same shape as they are physically. Its pretty darn easy to tell if there are shading problems. I've attached another graph of another panel thats shaded in the mornings by the chimney.
  • Peter_V
    Peter_V Solar Expert Posts: 226 ✭✭✭
    Re: Enphase Data Access
    andyman wrote: »
    They use openvpn to create a tunnel back to corporate. Even if you intercepted with dns redirection you don't have the private keys for the tunnel so pointless. All you'll do is secure the device which you can already do via firewalling the inside address..

    Well, that's why I said to replace the keys with your own keys.
    Anyway I've attached one of my graphs from my panel. Its in watts.
    That's pretty cool. I should get off my duff and try to duplicate your work.

    Thanks.
  • System2
    System2 Posts: 6,290 admin
    Re: Enphase Data Access

    Hope y'all don't mind me coming on board. I'm actually in the Los Angeles area. (But I do love to visit Sedona, Clarkdale and Jerome. I also spent an hour on the phone today with the Maricopa County Recorder.)

    I recently installed a 2.2 kW system on my house. As of last week, I have it all up and running with the Net Metering setup for my utility.

    I'm also curious about the enPhase data. I've tried a few things.

    I can see that the EMU provides a small HTTPD server for a web page.

    20110201_solar_production.png


    It also has - as someone mentioned earlier in the thread - an SSH connection. However, trying a few passwords lead to nothing.

    kai@proxy:~$ ssh root@192.168.0.100
    root@192.168.0.100's password:
    Permission denied, please try again.
    root@192.168.0.100's password:
    Permission denied, please try again.
    root@192.168.0.100's password:
    Permission denied (publickey,password,keyboard-interactive).


    Scraping the HTTP would be decent but not the best. I'm wondering if the SQLLite database holds any history beyond a current month.

    I see someone got at the passwd file. Was there any success in this?

    Oh, and I emailed enPhase yesterday to ask what the root login was.

    No word yet.
  • ag4wake
    ag4wake Registered Users Posts: 18
    Re: Enphase Data Access
    Scraping the HTTP would be decent but not the best. I'm wondering if the SQLLite database holds any history beyond a current month.

    in the Envoy manual and on the local website there is an indication of how much data the Envoy can hold without a connection to the internet. I don't have it in front of me, but I thought they said it can store 20 devices for 6months. It may have been 30 devices. I'll check again when I get home.
    I see someone got at the passwd file. Was there any success in this?

    I believe in this thread it was mentioned the passwd file md5 hashes were salted meaning it would take quite a while to brute force a break on it.

    Alan