the time i fucked the servers

Not literally sadly, so not actual nice fucking

but during the whole lockdown covid era we went full into remote work, and i moved from user support to networking and server infrastructure stuff

support was fine because mostly dealed with users and shit issues and got kinda paid the same, so i was moved to servers and well, i started working into that over the years, the boss i had at that point was a fucking piece of shit, an homophobic asshole, he left after a while and that work became mostly mine.

i had two big clusters to manage at that point, full prod and another one that i was building to replace it, iirc 3 dell servers with dumbass ram amounts, a couple of storage things with lots of ssd terabytes and stuff.

we ran the redhat virtualization suite, this was a long time ago, can't really remember the name of that, but we paid for it and whatever, but the main prod server ran the opensource version of that, so no support or whatever.

One day i noticed the webui of that server being slow, glitchy, i wasnt able to delete disks and stuff like that, considered it a fluke, reseted the web server got it back, but weirder.

big issue raised from there, if i turn off a vm, i wasnt able to get it back to run again, it just fucking dissapeared from existance.

we did had backups, but mostly of DBs, content, media, whatever, but the vms itself were never backed up.

i was kinda working on that, getting everythging prod running on the new nodes and then get a paid backup software.

well, fuck me, because this serever web interface was broken, attaching the new iscsi disks worked, and i was able to copy some disks and vms over, some failed, i moved some over a fucking thumbdrive, others to my computer and then to the new server

i found my ways, i cloned machine filesystems live over sftp, cloned windows VMs with clonezilla without rebooting.

at some point during the year i was close to finish, some of the most critical bullshit was still runing there, a haproxy that was routing some sites,  some wordpress sites

the whole shit was a mess, people before me created VMs for everything, one vm for one site, other for another, whatever, shit infraestructure.

one day, i was working on the secondary node remotely, i was running fedora and those servers ran centos, almost the same expertience over ssh and locally for me, got trusting and stuff.

i was fucking around outside my place during work hours and i was remoted to my computer in my phone

i was like im gonna reboot my computer

i rebooted one of the broken nodes

it was friday

because of course it was, fuck me.

i noticed what i did

and blood ran cold

i felt that shit feeling in my back

the server came back, some vms also did, but most were missing

shit was just gone to the ether idk

 

i just took an uber back home

while i was Omw (can you believe the mac tried to make that the fancy text replacement shit, hate those features) home and i saw the whatsapp blew up

they blamed the network guys, so i won some time, yeah fuck me i acted like "oh shit what happened im just a tiger lets go shopping"

 

most of the shit was down

 

and so it was, for like a some days

 

i made it to the office

 

boss was like wtf

and i was like the server is fucked yo

whats your plan

 

i was gonna be like, idk i quit fuck it

 

that was my though process the whole week i was fixing that.

 

so the thing is that the previous guy left the db of that webui running in a backup

so i reinstaled ovirt (remembered the name) and tried to restore that backup

error

error

invalid file

 

fuck me it was broken, i went months back in backups, yeah corrupt, it was just making a weird broken tar file, useless

 

so i was like idk what to do, but i attached the iscsi disks again, and i saw files appear on the disk explorer tyhing, no vms, no info, no nothing, but i started creating vms again, and bam, got one back running, i set up networking, vlans, other shit, got the server back

i did that for all the disk i found

 

create test check what it was

mostly it was fine

 

took some days, but got that shit running back again, and while i was doing that i was migrating VMs to the new fancy node

 

one day i was moving the last shit

 

hey one of the wordpress sites is broken, shows data but images are lost

 

yeah they were

 

gone, gonesky, missing, nada, ni una wea

 

backups?

lol

 

the backups had the database but no files for wordpress

 

fuck 

 

yeah we dont have those files anymore

backups are useless

the files probably were on a snapshot of that vm

and that is probably lost forever idk

 

 

so what do we do? what's your plan?

 

i was like: idk, im not the one that gonna take the hit for it, it's gonna be you so fuck it we roll i guess, if they have the files we can restore them but if not well we are fucked

 

i was done at that point, i wwas gonna quit

the "head" boss was an asshole, he though yelling would make shit work faster

 

are you gonna fire me? i asked and he was like nah

 

so fuck you, let me work in peace

 

after this the focus on this work changed for me

i focused on improve, certificate and get the fuck out of there

 

i walked some months later after that issue, maybe a bit less than a year, a november and i was like, you know, fuck it, i quit, im done

 

and i left, i had friends there, from before

support and telephony there was amazing tbh

 

i got my share and the lifetime experience working with analog phones, wiring, resolving stuff that was phisical work, running around, drilling shit, moving computers

the last bits of real computering work before it all went to shit

the morale of this history is that bosses can be assholes and you can get them fired

the morale is also that computer world sucks now, there is almost nothing good about it anymore, is just rotten,dead husk of what used to be

i pray to that last netware server running hidden behind a wall keeping a contability system still working, last remainings of that era

the dead phone system replaced by a cloud saas solution, pray for you little Nortel system, little panasonic system

computers are dead and so is working with them

fuck that

 

kiwa.