[Tracking Issue] Deep learning project #142

New issue

Open

opened 2024-10-09 12:45:18 +02:00 by rlahfa · 0 comments

rlahfa commented

2024-10-09 12:45:18 +02:00

Owner

We have some long-standing deep learning usecases with Whisper and others.

Now that krz01 is operational again and the Tesla K80 is installed, we are deploying various services and testing stability, here is the todo list:

Monitoring of nvidia-smi information
Nerfing of PL for the Tesla K80
PCIe passthrough in a VM to avoid the need to reboot the whole hypervisor all the time (requires probably >4GB decoding for MMIO and dealing with some BAR resizing issues)
vGPU exploration: a lot of unknowns
Whisper server via Faster Whisper (and not whisper.cpp): requires some fiddling to support Compute Capability 3.7 (our target for Tesla K80)
NGINX work for exposing the various API servers and putting them behind some OIDC authentication

Integrations:

Access to certain mailboxes (some GDPR & security work is required here)
Access to voice recording during meetings (Livekit has it integrated, Jitsi Meet could still do it via manual recording)
Automatic summarization of meetings notes

We have some long-standing deep learning usecases with Whisper and others. Now that krz01 is operational again and the Tesla K80 is installed, we are deploying various services and testing stability, here is the todo list: - [ ] Monitoring of nvidia-smi information - [ ] Nerfing of PL for the Tesla K80 - [ ] PCIe passthrough in a VM to avoid the need to reboot the whole hypervisor all the time (requires probably >4GB decoding for MMIO and dealing with some BAR resizing issues) - [ ] vGPU exploration: a lot of unknowns - [ ] Whisper server via Faster Whisper (and not whisper.cpp): requires some fiddling to support Compute Capability 3.7 (our target for Tesla K80) - [x] NGINX work for exposing the various API servers and putting them behind some OIDC authentication --- Integrations: - [ ] Access to certain mailboxes (some GDPR & security work is required here) - [ ] Access to voice recording during meetings (Livekit has it integrated, Jitsi Meet could still do it via manual recording) - [ ] Automatic summarization of meetings notes